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TIER  ONE  PERFORMANCE  SCREEN  INITIAL  OPERATIONAL 
TEST  AND  EVALUATION:  2012  ANNUAL  REPORT 

EXECUTIVE  SUMMARY 


Research  Requirement: 

In  addition  to  educational,  physical,  and  moral  screens,  the  U.S.  Army  relies  on  the  Armed  Forces 
Qualification  Test  (AFQT),  a  composite  score  from  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB),  to  select  new  Soldiers  into  the  Army.  Although  the  AFQT  has  proven  to  be  and 
will  continue  to  serve  as  a  useful  metric  for  selecting  new  Soldiers,  other  personal  attributes,  in 
particular  non-cognitive  attributes  (e.g.,  temperament,  interests,  and  values),  are  important  to  entry- 
level  Soldier  performance  and  retention  (e.g.,  Campbell  &  Knapp,  2001;  Ingerick,  Diaz,  &  Putka, 
2009;  Knapp  &  Heffner,  2009,  2010;  Knapp  &  Tremble,  2007).  Based  on  previous  research 
(Knapp  &  Heffner,  2010),  the  Army  selected  one  particularly  promising  measure,  the  Tailored 
Adaptive  Personality  Assessment  System  (TAP AS),  as  the  basis  for  an  initial  operational  test  and 
evaluation  (IOT&E)  of  the  Tier  One  Performance  Screen  (TOPS).  The  TAP  AS  capitalizes  on  the 
latest  advances  in  testing  technology  to  assess  motivation  through  the  measurement  of  personality 
characteristics. 

Procedure: 

In  May  2009,  the  Military  Entrance  Processing  Command  (MEPCOM)  began  administering  the 
TAPAS  on  the  computer  adaptive  platform  for  the  ASVAB  (CAT-ASVAB)  at  Military  Entrance 
Processing  Stations  (MEPS).  For  a  period  of  several  months,  the  Infonuation/Communications 
Technology  Literacy  (ICTL)  test  was  also  administered  to  Army  applicants.  To  evaluate  the 
TAPAS  and  ICTL,  outcome  (criterion)  data  are  being  collected  at  multiple  points  in  time  from 
Soldiers  who  took  the  TAPAS  at  entry.  Specifically,  initial  military  training  (IMT)  criterion  data  are 
being  collected  at  schools  for  Soldiers  in  eight  military  occupational  specialties  (MOS).  Project 
teams  are  also  collecting  criterion  data  from  Soldiers  (regardless  of  MOS)  in  their  units  in  multiple 
waves  of  site  visits  during  the  course  of  the  IOT&E. 

The  criterion  measures  include  job  knowledge  tests,  an  attitudinal  assessment  (the  Army  Life 
Questionnaire),  and  performance  rating  scales  completed  by  the  Soldiers’  cadre  members  (in  IMT)  or 
supervisors  (in  units).  Course  grades,  completion  rates,  and  attrition  status  are  obtained  from 
administrative  records  for  all  Soldiers. 

A  data  file  containing  TAPAS  data  collected  through  September  2012  and  criterion  data 
collected  through  December  2012  is  the  basis  for  the  analyses  documented  in  this  report.  It 
consists  of  a  total  of  344,953  applicants  who  took  the  TAPAS;  309,1 10  of  these  individuals  were 
in  the  TOPS  “Applicant  Sample.’’  The  Applicant  Sample  (used  for  analysis  purposes)  excluded 
Education  Tier  3,  AFQT  Category  V,  and  prior  service  applicants.  The  validation  sample  sizes 
were  considerably  smaller,  with  the  IMT  Validation  Sample  comprising  17,670  Soldiers,  the  In- 
Unit  Validation  Sample  comprising  1,053  Soldiers,  and  the  Administrative  Validation  Sample 
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(which  includes  Soldiers  with  criterion  data  [e.g.,  attrition]  from  at  least  one  administrative 
source)  comprising  141,170  Soldiers.  The  ICTL  Validation  Sample  comprises  1,758  Soldiers 
who  took  the  ICTL  when  it  was  being  administered  to  Army  applicants  from  May  2011  to 
January  2012. 

Data  from  the  job  knowledge  tests,  rating  scales,  attitudinal  assessment,  and  administrative 
sources  were  combined  to  yield  an  array  of  scores  representing  important  Soldier  outcomes.  In 
general,  the  criterion  scores  exhibited  acceptable  and  theoretically  consistent  psychometric 
properties.  The  exception  to  this  was  the  rating  scales,  which  continued  to  exhibit  low  inter-rater 
reliability.  Results  involving  the  rating  scales  should  continue  to  be  interpreted  with  caution. 

Our  approach  to  analyzing  the  TAPAS’  incremental  predictive  validity  was  consistent  with 
previous  evaluations  of  this  measure  and  similar  experimental  non-cognitive  predictors  (e.g., 
Ingerick  et  al.,  2009;  Knapp  &  Heffner,  2009,  2010,  2011).  In  brief,  this  approach  involved 
testing  a  series  of  hierarchical  regression  models,  regressing  scores  for  each  criterion  measure 
onto  Soldiers’  AFQT  scores  or  education  tier  in  the  first  step,  followed  by  their  TOPS  composite 
or  TAPAS  scale  scores  in  the  second  step.  The  resulting  increment  in  the  multiple  correlation 
value  (A R)  when  the  TOPS  composite  or  TAPAS  scale  scores  were  added  to  the  baseline 
regression  models  served  as  our  index  of  incremental  validity.  Scale-level  correlations  between 
TAPAS  scale  scores  and  selected  criteria  were  also  examined.  Analyses  used  the  original 
(operational  at  the  time  of  administration)  TOPS  Will-Do  and  Can-Do  composite  scores  as  well 
as  revised  Will-Do  and  Can-Do  composite  scores  plus  a  new  Adaptation  composite  score. 

Our  approach  to  analyzing  the  ICTL’s  predictive  and  discriminant  validity  was  consistent  with 
previous  evaluations  of  similar  experimental  non-cognitive  predictors,  however  we  focused  on 
Soldiers  in  five  MOS,  one  of  which  involves  cyber-related  job  duties.  The  approach  involved 
testing  a  series  of  hierarchical  regression  models,  regressing  scores  for  each  criterion  measure 
onto  Soldiers’  AFQT  scores  or  education  tier  in  the  first  step,  followed  by  their  ICTL  composite 
score  in  the  second  step.  The  resulting  increment  in  the  multiple  correlation  value  (A R)  when  the 
ICTL  composite  was  added  to  the  baseline  regression  models  served  as  the  index  of  incremental 
validity  for  the  measure.  Scale-level  correlations  between  the  ICTL  and  selected  criteria  were 
also  examined. 

Findings: 

Results  of  the  incremental  validity  analyses  indicate  that  the  TAPAS  predicts  important  first- 
term  criteria  over  and  above  the  AFQT,  especially  measures  tapping  non-technical  aspects  of 
Soldier  performance,  such  as  physical  fitness,  adjustment  to  Army  life,  commitment  and  fit,  and 
discipline.  The  revised  Will-Do  composite  was  associated  with  the  greatest  incremental  validity 
gains  compared  to  other  TOPS  composites.  This  was  especially  true  for  the  prediction  of  physical 
fitness.  None  of  the  TOPS  composites  demonstrated  utility  in  incrementing  the  AFQT  in  the 
prediction  of  attrition  up  to  30  months  in  service.  Results  of  the  previously  reported  classification 
analyses,  however,  indicated  that  the  TAPAS  has  the  potential  to  enhance  matching  new  Soldiers 
to  MOS,  particularly  for  minimizing  attrition. 
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Results  of  the  ICTL  validity  analyses  suggest  that  the  ICTL  test  is  a  valid  predictor  of  both  Can 
Do  and  Will  Do  performance  dimensions  across  both  cyber- focused  MOS  and  other  MOS. 
Attempts  to  examine  discriminant  validity  evidence  were  complicated  by  the  lack  of  MOS- 
specific  criterion  data  for  the  cyber-focused  MOS  (25B)  included  in  the  database.  Administration 
of  MOS-specific  criterion  measures  for  25B  Soldiers  began  well  after  the  IOT&E  began,  so  it 
will  take  more  time  to  get  sufficient  sample  sizes  to  support  discriminant  validity  analyses. 

Utilization  and  Dissemination  of  Findings: 

The  research  findings  will  be  used  by  the  Army  Deputy  Chief  of  Staff,  G-l;  U.S.  Army 
Recruiting  Command;  Assistant  Secretary  of  the  Army  (Manpower  and  Reserve  Affairs);  and 
Training  and  Doctrine  Command  to  evaluate  the  effectiveness  of  tools  used  for  Anny  applicant 
selection  and  assignment.  With  each  successive  set  of  findings,  the  TOPS  can  be  revised  and 
refined  to  meet  Army  needs  and  requirements. 
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TIER  ONE  PERFORMANCE  SCREEN  INITIAL  OPERATIONAL 
TEST  AND  EVALUATION:  2012  ANNUAL  REPORT 

CHAPTER  1:  INTRODUCTION 

Deirdre  J.  Knapp  (HumRRO),  Kate  LaPort,  Tonia  S.  Heffner,  and  Leonard  A.  White  (ARI) 


Background 

The  Personnel  Assessment  Research  Unit  (PARU)  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  is  responsible  for  conducting  personnel  research  for  the 
Anny.  The  focus  of  PARU’s  research  is  maximizing  the  potential  of  the  individual  Soldier 
through  effective  selection,  classification,  and  retention  strategies. 

In  addition  to  educational,  physical,  and  moral  screens,  the  U.S.  Army  relies  on  the  Anned 
Forces  Qualification  Test  (AFQT),  a  composite  score  from  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB),  to  select  new  Soldiers  into  the  Anny.  Although  the  AFQT  has 
proven  to  be,  and  will  continue  to  serve  as,  a  useful  metric  for  selecting  new  Soldiers,  other 
personal  attributes,  in  particular  non-cognitive  attributes  (e.g.,  temperament,  interests,  and 
values),  are  important  to  entry-level  Soldier  performance  and  retention  (e.g.,  Knapp  &  Tremble, 
2007). 

In  December  2006,  the  Department  of  Defense  (DoD)  ASVAB  review  panel — a  panel  of  experts 
in  the  measurement  of  human  characteristics  and  performance —  released  their  recommendations 
(Drasgow,  Embretson,  Kyllonen,  &  Schmitt,  2006),  several  of  which  focused  on  supplementing 
the  ASVAB  with  additional  measures  for  use  in  selection  and  classification  decisions.  The 
ASVAB  review  panel  further  recommended  that  the  use  of  these  measures  be  validated  against 
performance  criteria. 

Just  prior  to  the  release  of  the  ASVAB  review  panel’s  findings,  ARI  had  initiated  a  longitudinal 
research  effort,  Validating  Future  Force  Performance  Measures  (Army  Class),  to  examine  the 
prediction  potential  of  several  non-cognitive  measures  (e.g.,  temperament  and  person- 
environment  fit)  for  Anny  outcomes  (e.g.,  performance,  attitudes,  attrition).  The  Anny  Class 
research  project  was  a  6-year  effort  conducted  with  contract  support  from  the  Human  Resources 
Research  Organization  ([HumRRO];  Allen,  Knapp,  &  Owens,  in  preparation;  Ingerick,  Diaz,  & 
Putka,  2009;  Knapp  &  Heffner,  2009).  Experimental  predictors  were  administered  to  new 
Soldiers  in  2007  and  early  2008.  Anny  Class  collected  school-based  criterion  data  on  a  subset  of 
the  Soldier  sample  as  they  completed  job  training.  Job  perfonnance  criterion  data  were  collected 
from  Soldiers  in  the  Army  Class  longitudinal  validation  sample  in  2009  and  a  second  round  of 
“in-unit”  data  collections  was  completed  in  April  2011  (Knapp,  Owens,  &  Allen,  2012).  Final 
analysis  and  reporting  of  this  program  of  research  is  complete  (Allen,  Knapp,  &  Owens,  in 
preparation). 

After  the  Anny  Class  research  was  underway,  ARI  initiated  the  Expanded  Enlistment  Eligibility 
Metrics  (EEEM)  project  (Knapp  &  Heffner,  2010).  The  EEEM  goals  were  similar  to  Army 
Class,  but  the  focus  was  specifically  on  Soldier  selection  and  the  time  horizon  was  much  shorter. 
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Specifically,  EEEM  required  identification  of  one  or  more  promising  new  predictor  measures  for 
immediate  implementation.  The  EEEM  project  capitalized  on  the  existing  Anny  Class  data 
collection  procedure  and,  thus,  the  EEEM  sample  was  a  subset  of  the  Army  Class  sample. 

As  a  result  of  the  EEEM  findings,  Anny  policy-makers  approved  an  initial  operational  test  and 
evaluation  (IOT&E)  of  the  Tier  One  Performance  Screen  (TOPS).  This  report  is  the  sixth  in  a 
series  presenting  continuing  analyses  from  the  IOT&E  of  TOPS. 

The  Tier  One  Performance  Screen  (TOPS) 

Six  experimental  pre-enlistment  measures  were  included  in  the  EEEM  research  (Allen,  Cheng, 
Putka,  Hunter,  &  White,  2010).  These  included  several  temperament  measures,  a  situational 
judgment  test,  and  two  person-environment  fit  measures  based  on  values  and  interests.  The  most 
promising  measures  recommended  to  the  Anny  for  implementation  were  identified  based  on  the 
following  considerations: 

•  Incremental  validity  over  AFQT  for  predicting  important  performance  and  retention- 
related  outcomes 

•  Minimal  subgroup  differences 

•  Low  susceptibility  to  response  distortion  (e.g.,  faking  optimal  responses) 

•  Minimal  administration  time  requirements 

The  Tailored  Adaptive  Personality  Assessment  System  ([TAPAS];  Stark,  Chernyshenko,  & 
Drasgow,  2010)  surfaced  as  the  top  choice,  with  the  Work  Preferences  Assessment  ([WPA]; 
Putka  &  Van  Iddekinge,  2007)  identified  as  another  good  option  that  was  substantively  different 
from  the  TAPAS.  Specifically,  the  TAPAS  is  a  measure  of  personality  characteristics  (e.g., 
achievement,  sociability)  that  capitalizes  on  the  latest  advances  in  psychometric  theory  and 
provides  a  good  indicator  of  personal  motivation.  The  WPA  asks  applicants  to  indicate  their 
preference  for  various  kinds  of  work  activities  and  environments  (e.g.,  “A  job  that  requires  me  to 
teach  others,”  “A  job  that  requires  me  to  work  outdoors”).  Although  not  included  in  the  EEEM 
research,  the  Information/  Communications  Technology  Literacy  (ICTL)  test  emerged  as  a 
potential  test  of  applicants’  familiarity  with  computers  and  infonnation  technology,  which  may 
predict  performance  in  high-technology  occupations  (Russell  &  Sellman,  2009). 

In  May  2009,  the  Military  Entrance  Processing  Command  (MEPCOM)  began  administering  the 
TAPAS  on  the  computer  adaptive  platform  for  the  ASVAB  (CAT-ASVAB).  Initially,  the 
TAPAS  was  to  be  administered  only  to  Education  Tier  1,  non-prior  service  applicants. 1  This 
limitation  to  Education  Tier  1  was  removed  early  in  CY201 1  so  the  Anny  could  evaluate  the 
TAPAS  across  all  types  of  applicants. 

TOPS  uses  non-cognitive  measures  to  identify  applicants  who  would  likely  perfonn  differently 
(higher  or  lower)  than  would  be  predicted  by  their  ASVAB  scores.  As  part  of  the  TOPS  IOT&E, 
TAPAS  scores  are  being  used  to  screen  out  a  small  number  of  AFQT  Category  TUB  /TV 


1  Applicant  educational  credentials  are  classified  as  Tier  1  (primarily  high  school  diploma),  Tier  2  (primarily  non¬ 
diploma  graduate),  and  Tier  3  (not  a  high  school  graduate). 
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applicants."  Although  the  WPA  is  part  of  the  TOPS  IOT&E,  WPA  scores  will  not  be  considered 
for  enlistment  eligibility.  The  WPA  is  scheduled  for  administration  at  MEPS  starting  in  late 
CY2013. 


Although  the  initial  conceptualization  for  the  IOT&E  was  to  use  the  TAPAS  as  a  tool  for 
“screening  in”  Education  Tier  1  applicants  with  lower  AFQT  scores,  changing  economic 
conditions  spurred  a  reconceptualization  that  led  to  using  the  TAPAS  as  a  tool  that  screens  out 
low  motivated  applicants.  Recruiting  conditions  continue  to  shift,  so  both  the  IOT&E  and  any 
subsequent  fully  operational  system  will  need  to  adjust  to  fit  with  the  applicant  market.  TAPAS 
composite  scores  and  cut  points  can  be  modified  as  needed  to  fit  recruiting  market  conditions. 

Evaluating  TOPS 

To  evaluate  the  pre-enlistment  measures  (TAPAS,  WPA,  and  ICTL),  the  Army  is  collecting 
training  criterion  data  on  Soldiers  in  eight  target  military  occupational  specialties  (MOS)  as  they 
complete  initial  military  training  (IMT).2 3  The  criterion  measures  include  job  knowledge  tests 
(JKTs);  an  attitudinal  assessment,  the  Anny  Life  Questionnaire  (ALQ);  and  perfonnance  rating 
scales  (PRS)  completed  by  the  Soldiers’  cadre.  These  measures  are  computer-administered  at  the 
schools  (initial  military  training)  for  each  of  the  eight  target  MOS.  The  process  is  overseen  by 
Anny  personnel  with  guidance  and  support  from  both  ARI  and  HumRRO.  Course  grades  and 
completion  rates  are  obtained  from  administrative  records  for  all  Soldiers  who  take  the  TAPAS, 
regardless  of  MOS. 

Criterion  data  are  also  being  collected  from  Soldiers  and  their  supervisors  during  data  collection 
trips  to  major  Anny  installations.  These  proctored  “in-unit”  data  collections  began  in  January 
2011  and  target  all  Soldiers  who  took  the  TAPAS  prior  to  enlistment.  The  in-unit  criterion 
measures  include  JKTs,  the  ALQ  attitudinal  assessment,  and  supervisor  ratings  of  performance. 
The  data  collection  model  closely  mirrors  that  which  was  used  in  the  Anny  Class  research 
program  (Knapp  et  al.,  2012).  Separation  status  of  all  Soldiers  who  took  the  TAPAS  prior  to 
enlistment  is  tracked  throughout  the  course  of  the  research. 

This  report  describes  the  sixth  iteration  of  developing  a  criterion-related  validation  data  file  and 
conducting  evaluation  analyses  using  data  collected  in  the  TOPS  IOT&E  initiative.  Prior 
evaluations  are  described  in  a  series  of  technical  reports  (Knapp  &  Heffner,  2011,  2012;  Knapp, 
Heffner,  &  White,  2011;  Knapp  &  LaPort,  2013a,  2013b).  Additional  analysis  datasets  and 
validation  analyses  will  be  prepared  and  conducted  at  6-month  intervals  throughout  the  multi¬ 
year  IOT&E  period.  For  the  first  time,  the  current  evaluation  includes  results  related  to  the  ICTL 
test. 


2  Examinees  are  classified  into  categories  based  on  their  AFQT  percentile  scores  (Category  I  =  93-99,  Category  II  = 
65-92,  Category  IIIA  =  50-64,  Category  IIIB  =  31-49,  Category  IV  =  10-30,  Category  V  =  1-9). 

3  The  target  MOS  are  Infantryman  ( 1  IB),  Armor  Crewman  (19K),  Signal  Support  Specialist  (25U),  Military  Police 
(3  IB),  Human  Resources  Specialist  (42A),  Health  Care  Specialist  (68W),  Motor  Transport  Operator  (88M),  and 
Light  Wheel  Vehicle  Mechanic  (91B).  These  MOS  were  selected  to  include  large,  highly  critical  MOS  as  well  as  to 
represent  the  diversity  of  work  requirements  across  MOS. 
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Overview  of  Report 


Chapter  2  explains  how  the  evaluation  analysis  data  files  are  constructed  and  then  describes 
characteristics  of  the  samples  resulting  from  construction  of  the  latest  analysis  data  file.  Chapter 
3  describes  the  TAPAS  and  ASVAB,  including  content,  scoring,  and  psychometric 
characteristics.  Chapter  4  describes  the  IMT  and  in-unit  criterion  scores  used  in  this  evaluation, 
including  their  psychometric  characteristics.  Criterion-related  validation  analyses  for  the  TAPAS 
are  presented  in  Chapter  5.  Chapter  6  describes  the  ICTL  test,  its  psychometric  properties,  and 
criterion-related  validation  results.  The  report  concludes  with  Chapter  7,  which  summarizes  our 
continuing  efforts  to  evaluate  TOPS  and  looks  toward  plans  for  future  iterations  of  these 
evaluations. 
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CHAPTER  2:  DATA  FILE  DEVELOPMENT 


D.  Matthew  Trippe,  Bethany  Bynum,  Karen  Moriarty,  and  Chad  Peddie  (HumRRO) 


Overview  of  Process 

The  TOPS  data  file  comprises  predictor  and  criterion  data  obtained  from  administrative,  IMT, 
and  in-unit  sources.  The  IMT  and  in-unit  assessments  are  described  in  Chapter  4. 

An  illustrative  view  of  the  TOPS  analysis  file  construction  process  is  provided  in  Figure  2.1. 4 
The  lighter  boxes  within  the  figure  represent  source  data  files,  and  the  darker  boxes  represent 
samples  on  which  descriptive  or  inferential  analyses  are  conducted.  Samples  are  fonned  by 
applying  filters  to  a  data  file  such  that  it  includes  the  observations  of  interest.  The  leftmost 
column  in  the  figure  summarizes  the  predictor  data  sources  used  to  derive  the  TOPS  Applicant 
Sample.  The  other  columns  summarize  the  research-only  (i.e.,  non-administrative)  and 
administrative  criterion  data.  Predictor  and  criterion  data  are  merged  to  form  the  IMT  or  in-unit 
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Figure  2.1.  Overview  of  TOPS  data  file  merging  and  nested  sample  generation  process. 


4  Administrative  data  are  collected  from  the  following  sources:  (a)  Defense  Manpower  Data  Center  (DMDC),  (b)  Army 
Human  Resources  Command  (AHRC),  (c)  Army  Training  Support  Center’s  (ATSC)  Resident  Individual  Training 
Management  System  (RITMS),  and  (d)  Training  and  Doctrine  Command’s  (TRADOC)  Army  Training  Requirements  and 
Resources  System  (ATRRS). 
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validation  samples  and  the  large  administrative  validation  sample,  which  includes  all  Soldiers 
who  have  predictor  data  and  at  least  one  criterion  record  (e.g.,  administrative  data).  The  latest 
version  of  the  TOPS  data  file  does  not  contain  WPA  predictor  scores  since  that  measure  is  not 
yet  being  administered  to  Army  applicants. 

Description  of  Data  File  and  Sample  Construction 

The  latest  data  file,  created  in  December  2012,  includes  TAPAS  data  collected  from  May  2009 
through  September  2012  and  criterion  data  collected  through  December  2012.  Table  2.1 
summarizes  the  relevant  characteristics  of  the  total  TAPAS  sample  contained  in  the  December 
2012  TOPS  data  file.  The  total  sample  includes  applicants  who  did  not  enlist  in  the  Anny.  The 
TOPS  Applicant  Sample  was  defined  by  limiting  records  in  the  total  sample  data  file  provided  by 
MEPCOM  to  those  Soldiers  who  are  non-prior  service,  Education  Tier  1  or  25,  and  have  an 
AFQT  score  of  10  or  greater.  Among  the  344,953  applicants  in  the  total,  unfiltered  sample, 

309,1 10  (89.6%)  met  these  screens  and  constituted  the  Applicant  Sample. 

Sample  sizes  reported  in  all  subsequent  chapters  and  appendices  will  generally  be  smaller  than 
the  initial  numbers  reported  here  because  of  further  data  filtering  or  disaggregation  that  occurs 
for  each  particular  analysis.  Predictor  and  criterion  scores  were  determined  to  be  valid  if  they 
passed  multiple  data  quality  screens  intended  to  identify  unmotivated  applicants.  Those 
additional  screens  have  not  yet  been  applied  to  the  samples  described  in  this  chapter  because 
they  are  often  specific  to  a  particular  analysis.  Further,  a  relatively  small  number  of  Soldiers 
(1,646)  in  the  Applicant  Sample  who  were  administered  an  early  version  of  the  TAPAS  were 
excluded  from  analyses  because  of  conceptual  dissimilarities  with  subsequent  TAPAS  forms. 

A  detailed  breakout  of  background  and  demographic  characteristics  observed  in  the  analytic 
samples  appears  in  Table  2.2.  Regular  Anny  Soldiers  comprise  a  majority  of  the  cases  in  each 
sample.  The  samples  are  predominantly  male,  Caucasian,  and  non-Hispanic;  however,  a  large 
percentage  of  Soldiers  declined  to  provide  information  on  race  or  ethnicity. 

The  Administrative  Validation  Sample  described  in  Table  2.2  includes  141,170  Soldiers. 

Included  in  this  sample  are  Soldiers  who  meet  all  of  the  inclusion  criteria  for  the  TOPS 
Applicant  Sample  and  also  have  at  least  one  record  in  an  administrative  criterion  data  source 
(i.e.,  Army  Training  Requirements  and  Resources  System  [ATRRS],  Resident  Individual 
Training  Management  System  [RITMS],  attrition).  However,  the  number  of  Soldiers  included  in 
any  individual  analysis  is  generally  much  smaller.  The  exact  number  of  Soldiers  varies  by 
criterion  depending  on  the  availability  of  valid  data  on  key  variables.  Specific  sample  details  on 
each  criterion  variable  are  provided  in  subsequent  chapters. 

Although  there  are  52,606  Soldiers  in  the  Full  IMT  data  file,  only  17,670  had  taken  the  TAPAS 
when  they  applied  for  enlistment.  There  are  two  primary  reasons  for  this  disconnect.  First,  early 
in  the  research  effort  most  of  the  Soldiers  tested  at  the  schools  had  taken  their  pre-enlistment 


5  Starting  with  the  June  2012  TOPS  data  file,  we  incorporated  education  tier  information  from  a  AHRC  data  source 
to  best  capture  a  Soldier’s  education  tier  status  at  the  time  of  his  or  her  accession.  As  a  result,  figures  for  education 
tier  reported  in  the  current  report  will  differ  from  corresponding  figures  in  previous  reports.  The  differences  were 
generally  minor  and  did  not  impact  the  overall  results  or  findings. 


6 


tests  before  MEPCOM  started  administering  the  TAPAS  widely  to  applicants.  Second,  we  rely 
on  name  and  date  of  birth  to  match  TAPAS  records  to  the  criterion  data,  which  often  results  in 
unsuccessful  matches.  As  expected,  the  analysis  data  files  have  shown  progressively  higher 
match  rates  between  Soldiers  tested  in  the  schools  and  those  tested  pre-enlistment.  The  overall 
match  rate  at  this  stage  (33.5%)  compares  to  5.5%  in  the  first  semi-annual  evaluation  cycle 
(Trippe,  Ford,  Bynum,  &  Moriarty,  2012).  The  match  rate  for  new  cases  added  this  cycle  was 
66.8%.  Similarly,  there  are  3,780  Soldiers  with  in-unit  data  but  only  1,053  of  these  Soldiers  have 
matching  TAPAS  data.  There  are  189  Soldiers  with  a  TAPAS  record  and  both  IMT  and  in-unit 
criterion  data. 


Table  2.1.  Full  TAPAS  Data  File  Characteristics 


Variables 

n 

%  of  Total  Sample 
(N  =  344,953.) 

Education  Tier 

Tier  1 

320,593 

92.9 

Tier  2 

18,479 

5.4 

Tier  3 

5,876 

1.7 

Unknown 

5 

0.0 

Prior  Service 

Yes 

8,471 

2.5 

No  or  Missing 

336,482 

97.5 

Military’  Occupational  Specialty 

1  IB/1 1C/1 1X/18X 

29,069 

8.4 

19K 

1,597 

0.5 

25U 

2,884 

0.8 

31B 

7,625 

2.2 

42A 

4,154 

1.2 

68W 

9,011 

2.6 

88M 

8,764 

2.5 

91B 

8,270 

2.4 

Other 

102,879 

29.8 

Unknown 

170,700 

49.5 

AFQT  Category 

I 

22,072 

6.4 

II 

97,623 

28.3 

IIIA 

65,959 

19.1 

IIIB  b 

104,186 

30.2 

IV  b 

49,439 

14.3 

V 

5,662 

1.6 

Contract  Status 

Signed 

211,418 

61.3 

Not  signed 

133,535 

38.7 

Applicant  Sample  c 

309,110 

89.6 

“Generally,  when  the  MOS  is  unknown,  it  is  either  because  the  respondent  did  not  access  into  the  Army  or  because  the 
information  was  not  yet  available  in  the  data  sources  on  which  the  Dec  2012  data  file  was  based. 
b  AFQT  Category  IIIB  and  IV  is  oversampled.  Figures  presented  are  not  representative  of  Army  accessions. 
c  The  Applicant  Sample  size  is  smaller  than  the  total  TAPAS  sample  because  it  is  limited  to  non-prior  service,  Education  Tier  1 
and  2,  and  AFQT  >  10  applicants. 


7 


Table  2.2.  Background  and  Demographic  Characteristics  of  the  TOPS  Samples 


Applicant a 

77  =  309,110 

Administrative 
Validation  b 

77  =  141,170 

IMT 

Validation  c 

77  =  17,670 

In- Unit 
Validation  d 
n  =  1,053 

Characteristic 

n 

% 

n 

% 

n 

% 

n 

% 

Component 

Regular 

181,358 

58.7 

81,064 

57.4 

10,585 

59.9 

1,049 

99.6 

ARNG 

88,552 

28.6 

41,122 

29.1 

5,248 

29.7 

- 

- 

USAR 

39,200 

12.7 

18,966 

13.4 

1,837 

10.4 

- 

- 

Unknown 

— 

— 

18 

0.0 

- 

- 

4 

.4 

Education  Tier 

Tier  1 

292,165 

94.5 

135,833 

96.2 

17,011 

96.3 

1,031 

97.9 

Tier  2 

16,945 

5.5 

5,337 

3.8 

659 

3.7 

22 

2.1 

Military  Occupational  Specialty 

1  IB/1 1C/1 1X/18X  26,220 

8.5 

23,810 

16.9 

7,187 

40.7 

180 

17.1 

19K 

1,480 

0.5 

1,296 

0.9 

353 

2.0 

19 

1.8 

25U 

2,692 

0.9 

2,260 

1.6 

12 

0.1 

13 

1.2 

31B 

6,932 

2.2 

6,020 

4.3 

2,816 

15.9 

34 

3.2 

42A 

3,791 

1.2 

3,136 

2.2 

465 

2.6 

37 

3.5 

68W 

8,349 

2.7 

7,371 

5.2 

3,212 

18.2 

44 

4.2 

88M 

7,998 

2.6 

6,549 

4.6 

2,732 

15.5 

56 

5.3 

91B 

7,513 

2.4 

6,320 

4.5 

428 

2.4 

57 

5.4 

Other 

93,821 

30.4 

84,197 

59.6 

465 

2.6 

613 

58.2 

Unknown 

150,314 

48.6 

211 

0.1 

— 

— 

— 

— 

AFQT  Category’ 

1 

19,758 

6.4 

10,397 

7.4 

1,346 

7.6 

67 

6.4 

11 

88,826 

28.7 

47,382 

33.6 

6,801 

38.5 

320 

30.4 

II1A 

60,156 

19.5 

30,670 

21.7 

3,748 

21.2 

224 

21.3 

11IB 

95,310 

30.8 

45,835 

32.5 

5,027 

28.4 

397 

37.7 

IV 

45,060 

14.6 

6,886 

4.9 

748 

4.2 

45 

4.3 

Gender 

Female 

62,076 

20.1 

24,032 

17.0 

2,308 

13.1 

163 

15.5 

Male 

243,091 

78.6 

115,524 

81.8 

15,195 

86.0 

886 

84.1 

Race 

African  American 

59,917 

19.4 

23,583 

16.7 

2,243 

12.7 

220 

20.9 

American  Indian 

2,392 

0.8 

1,001 

0.7 

141 

0.8 

5 

0.5 

Asian 

9,965 

3.2 

4,743 

3.4 

542 

3.1 

41 

3.9 

Hawaiian/Pacific  Islander 

1,544 

0.5 

751 

0.5 

97 

0.5 

6 

0.6 

Caucasian 

221,156 

71.5 

106,176 

75.2 

13,889 

78.6 

733 

69.6 

Multiple 

1,254 

0.4 

619 

0.4 

81 

0.5 

2 

0.2 

Declined  to  Answer 

12,857 

4.2 

4,277 

3.0 

675 

3.8 

46 

4.4 

Ethnicity 

Hispanic/Latino 

46,739 

15.1 

20,221 

14.3 

2,160 

12.2 

127 

12.1 

Not  Hispanic 

250,022 

80.9 

117,308 

83.1 

14,931 

84.5 

891 

84.6 

Declined  to  Answer 

12,330 

4.0 

3,627 

2.6 

578 

3.3 

35 

3.3 

a  Limited  to  applicants  who  had  no  prior  service.  Education  Tier  1  or  2,  and  AFQT  >  10;  served  as  the  core  analysis  sample. 
b  Soldiers  in  Applicant  Sample  with  at  least  one  criterion  record  (i.e.,  schoolhouse,  in-unit,  ATRRS,  RITMS,  or  attrition). 
c  Soldiers  in  Applicant  Sample  with  criterion  data  collected  at  schoolhouses. 
d  Soldiers  in  Applicant  Sample  with  criterion  data  collected  in  units. 


Summary 


The  TOPS  data  file  is  periodically  updated  by  merging  new  TAPAS  scores,  administrative 
records,  IMT,  and  in-unit  data  into  one  master  data  file.  The  December  2012  data  file  includes  a 
total  of  344,953  applicants  who  took  the  TAPAS.  Of  these,  309,1 10  were  in  the  TOPS  Applicant 
Sample.  The  Applicant  Sample  was  detennined  by  excluding  Education  Tier  3,  AFQT 
Category  V,  and  prior  service  applicants  from  the  master  data  file.  Of  that  Applicant  Sample, 
141,170  (45.7%)  had  a  record  in  at  least  one  of  the  administrative  criterion  data  sources;  17,670 
had  IMT  data  collected  from  the  schoolhouse  and  1,053  had  in-unit  criterion  data.  Although 
subsequent  iterations  of  the  TOPS  IOT&E  data  file  will  have  progressively  larger  sample  sizes  to 
support  validation  and  other  evaluative  analyses,  these  sample  sizes  are  sufficient  to  warrant 
reasonable  confidence  in  the  evaluation  results. 
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CHAPTER  3:  DESCRIPTION  OF  THE  PRIMARY  TOPS  IOT&E  PREDICTOR 

MEASURES 

Stephen  Stark,  O.  Sasha  Chernyshenko,  Fritz  Drasgow  (Drasgow  Consulting  Group),  and 

Deirdre  J.  Knapp  (HumRRO) 

The  purpose  of  this  chapter  is  to  describe  the  primary  predictor  measures  being  investigated  in 
the  TOPS  IOT&E  (TAPAS  and  ASVAB).  The  central  predictor  under  investigation  in  this 
analysis  is  the  TAPAS  (Drasgow,  Stark,  Chernyshenko,  Nye,  Hulin,  &  White,  2012;  Stark  et  ah, 
2010),  while  the  baseline  predictor  used  by  the  Army  is  the  ASVAB.  Another  experimental 
predictor,  the  ICTL  (Russell  &  Sellman,  2009),  is  described  further  in  Chapter  6  along  with  a 
presentation  of  evaluation  results.  Data  on  the  final  experimental  predictor,  the  Work  Preferences 
Assessment  (WPA;  Putka  &  Van  Iddekinge,  2007),  are  not  yet  included  in  the  analysis  data  files, 
and  are  therefore  not  discussed  further  in  this  report. 

Tailored  Adaptive  Personality  Assessment  System  (TAPAS) 

Description 

The  TAPAS  is  a  personality  measurement  tool  originally  developed  by  Drasgow  Consulting  Group 
(DCG)  under  the  Army’s  Small  Business  Innovation  Research  (SBIR)  program.  The  system  builds 
on  the  foundational  work  of  the  Assessment  of  Individual  Motivation  ([AIM];  White  &  Young, 

1998)  by  incorporating  features  designed  to  promote  resistance  to  faking  and  by  measuring  narrow 
personality  constructs  (i.e.,  facets)  that  are  known  to  predict  outcomes  in  work  settings.  Because  the 
TAPAS  uses  item  response  theory  (IRT)  methods  to  construct  and  score  items,  it  can  be  administered 
in  multiple  fonnats:  (a)  as  a  fixed  length,  non-adaptive  test  where  examinees  respond  to  the  same 
sequence  of  items  or  (b)  as  an  adaptive  test  where  each  examinee  responds  to  a  unique  sequence  of 
items  selected  to  maximize  measurement  accuracy  for  that  specific  examinee. 

The  TAPAS  uses  an  IRT  model  for  multidimensional  pairwise  preference  items  ([MUPP];  Stark, 
Chernyshenko,  &  Drasgow,  2005)  as  the  basis  for  constructing,  administering,  and  scoring 
personality  tests  that  are  designed  to  reduce  response  distortion  (i.e.,  faking)  and  yield  normative 
scores  even  with  tests  of  high  dimensionality  (Stark,  Chernyshenko,  &  Drasgow  2012).  TAPAS 
items  consist  of  pairs  of  personality  statements  for  which  a  respondent’s  task  is  to  choose  the  one 
that  is  “more  like  me.”  The  two  statements  constituting  each  item  are  matched  in  terms  of  social 
desirability  and  often  represent  different  dimensions.  As  a  result,  it  is  difficult  for  respondents  to 
discern  which  answers  improve  their  chances  of  being  enlistment  eligible.  Because  they  are  less 
likely  to  know  which  dimensions  are  being  used  for  selection,  they  are  less  likely  to  identify 
which  statements  measure  those  dimensions,  and  they  are  less  likely  to  be  able  to  keep  track  of 
their  answers  on  several  dimensions  simultaneously  so  as  to  provide  consistent  patterns  of 
responses  across  the  whole  test.  Without  knowing  which  answers  have  an  impact  on  their 
eligibility  status,  respondents  should  not  be  able  to  increase  their  scores  on  selection  dimensions 
as  easily  as  when  traditional,  single  statement  measures  are  used.  In  short,  the  TAPAS’  features 
make  it  difficult  for  applicants  to  distort  their  responses  to  obtain  more  desirable  scores. 
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The  use  of  a  formal  IRT  model  also  greatly  increases  the  flexibility  of  the  assessment  process.  A 
variety  of  test  versions  can  be  constructed  to  measure  personality  dimensions  that  are  relevant  to 
specific  work  contexts,  and  the  measures  can  be  administered  via  paper-and-pencil  or 
computerized  fonnats.  If  test  content  specifications  (i.e.,  test  blueprints)  are  comparable  across 
versions,  the  respective  scores  can  be  readily  compared  because  the  metric  of  the  statement 
parameters  has  already  been  established  by  calibrating  response  data  obtained  from  a  base  or 
reference  group  (e.g.,  Army  recruits).  The  same  principle  applies  to  adaptive  testing,  wherein 
each  examinee  receives  a  different  set  of  items  chosen  specifically  to  reduce  the  error  in  his  or 
her  trait  scores  at  points  throughout  the  exam.  Adaptive  item  selection  enhances  test  security 
because  there  is  less  overlap  across  examinees  in  terms  of  the  items  presented. 

Another  important  feature  of  the  TAPAS  is  that  pools  of  statements  representing  more  than  20 
narrow  personality  traits  are  available.  The  initial  TAPAS  trait  taxonomy  was  developed  using  the 
results  of  several  large  scale  factor-analytic  studies  with  the  goal  of  identifying  a  comprehensive  set 
of  non-redundant  narrow  traits.  Since  the  TAPAS  was  initially  developed,  additional  traits  have  been 
added.  These  narrow  traits,  if  necessary  or  desired,  can  be  combined  to  form  either  the  Big  Five  (the 
most  common  organization  scheme  for  narrow  personality  traits)  or  any  other  number  of  broader 
traits  (e.g..  Integrity  or  Positive  Core  Self-Evaluations).  This  is  advantageous  for  applied  purposes 
because  TAPAS  versions  can  be  created  to  fit  a  wide  range  of  applications  and  are  not  limited  to  a 
particular  service  branch  or  criterion.  Selection  of  specific  TAPAS  dimensions  can  be  guided  by 
consulting  the  results  of  a  meta-analytic  study  perfonned  by  DCG  that  mapped  22  TAPAS 
dimensions  to  several  important  organizational  criteria  for  military  and  civilian  jobs  (e.g.,  task 
proficiency,  training  perfonnance,  attrition)  (Chernyshenko  &  Stark,  2007),  as  well  as  subsequent 
validation  research. 

Scoring  details  and  the  criterion-related  validation  work  that  led  to  the  inclusion  of  TAPAS  in 
the  TOPS  IOT&E  can  be  found  in  the  Expanded  Enlistment  Eligibility  Metrics  report  (Knapp  & 
Heffner,  2010)  and  in  earlier  evaluation  reports  in  this  series  (Knapp  et  ah,  201 1;  Knapp  & 
Heffner,  2011) 


Multiple  Versions  of  TAPAS 

As  part  of  the  TOPS  IOT&E,  multiple  versions  of  the  TAPAS  have  been  administered  as  ARI 
explores  the  value  of  new  and  alternative  dimensions  (see  Table  3.1  for  a  list  of  dimension  names 
and  descriptions.)  One  version  was  nonadaptive  (static),  so  all  examinees  answered  the  same 
sequence  of  items;  the  others  were  adaptive,  so  each  examinee  answered  items  tailored  to  his  or  her 
trait  level  estimates.  The  15D-Static  TAPAS  fonn  was  administered  from  mid- July  to  mid- 
September  of 2009  to  all  examinees,  and  later  to  smaller  numbers  of  examinees  at  some  MEPS.  The 
initial  adaptive  version  ( 15D-CAT-1 )  was  introduced  in  September  2009  and  included  the  same  15 
dimensions.  In  August  2011,  three  new  15-dimension  adaptive  versions  of  the  TAPAS  were 
introduced  into  the  MEPS  ( 15D-CAT-2 ,  Fonns  A,  B,  and  C)  to  replace  the  original  versions.  All 
TAPAS  fonns  used  in  the  IOT&E  assess  the  same  nine  core  dimensions,  to  include  all  of  the  scales 
in  the  TOPS  first  operational  “can-do”  and  “will-do”  composites  (described  next).  Each  15D  fonn 
also  includes  six  of  12  experimental  dimensions.  The  six  experimental  dimensions  assessed  vary  by 
fonn.  Note  also  that  the  Version  2  fonns  of  TAPAS  use  statement  pools  that  were  created 
exclusively  for  ARI.  In  the  present  report,  the  validation  analyses  reported  in  Chapter  5  are  based  on 
the  five  15-D  versions  of  TAPAS,  each  administering  120  items  (i.e.,  pairs  of  statements). 
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Table  3.1.  TAPAS  Dimensions  Names  and  Definitions 


Facet  Name 

Brief  Description 

Achievement 

High  scoring  individuals  are  seen  as  hard  working,  ambitious,  confident,  and 
resourceful. 

Adjustment 

High  scoring  individuals  are  well  adjusted,  worry  free,  and  handle  stress  well. 

Adventure  Seeking 

High  scoring  individuals  enjoy  participating  in  extreme  sports  and  outdoor 
activities. 

Aesthetics 

High  scoring  individuals  appreciate  various  forms  of  art  and  music  and 
participate  in  art-related  activities  more  than  most  people. 

Attention  Seeking 

High  scoring  individuals  tend  to  engage  in  behaviors  that  attract  social 
attention.  They  are  loud,  loquacious,  entertaining,  and  even  boastful. 

Commitment  to  Serve 

High  scoring  individuals  identify  with  the  military  and  have  a  strong  desire  to 
serve  their  country. 

Consideration 

High  scoring  individuals  are  affectionate,  compassionate,  sensitive,  and  caring. 

Cooperation 

High  scoring  individuals  are  pleasant,  trusting,  cordial,  non-critical,  and  easy 
to  get  along  with. 

Courage 

High  scoring  individuals  stand  up  to  challenges  and  are  not  afraid  to  face 
dangerous  situations. 

Curiosity 

High  scoring  individuals  are  inquisitive  and  perceptive;  they  are  interested  in 
learning  new  information  and  attend  courses  and  workshops  whenever  they 
can. 

Dominance 

High  scoring  individuals  are  domineering,  “take  charge”  and  are  often  referred 
to  by  their  peers  as  "natural  leaders." 

Even  Tempered 

High  scoring  individuals  tend  to  be  calm  and  stable.  They  don’t  often  exhibit 
anger,  hostility,  or  aggression. 

Ingenuity 

High  scoring  individuals  are  inventive  and  can  think  "outside  of  the  box." 

Intellectual  Efficiency 

High  scoring  individuals  believe  they  process  information  and  make  decisions 
quickly;  they  see  themselves  (and  they  may  be  perceived  by  others)  as 
knowledgeable,  astute,  or  intellectual. 

Non-Delinquency 

High  scoring  individuals  tend  to  comply  with  rales,  customs,  norms,  and 
expectations,  and  they  tend  not  to  challenge  authority. 

Optimism 

High  scoring  individuals  have  a  positive  outlook  on  life  and  tend  to 
experience  joy  and  a  sense  of  well-being. 

Order 

High  scoring  individuals  tend  to  organize  tasks  and  activities  and  desire  to 
maintain  neat  and  clean  surroundings. 

Physical  Conditioning 

High  scoring  individuals  tend  to  engage  in  activities  to  maintain  their  physical 
fitness  and  are  more  likely  participate  in  vigorous  sports  or  exercise. 

Responsibility 

High  scoring  individuals  are  dependable,  reliable,  and  make  every  effort  to 
keep  their  promises. 
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Table  3.1.  (Continued) 


Facet  Name 

Brief  Description 

Self  Control 

High  scoring  individuals  tend  to  be  cautious,  levelheaded,  able  to  delay 
gratification,  and  patient. 

Selflessness 

High  scoring  individuals  are  generous  with  their  time  and  resources. 

Situational  Awareness 

High  scoring  individuals  pay  attention  to  their  surroundings  and  rarely  get  lost 
or  surprised. 

Sociability 

High  scoring  individuals  tend  to  seek  out  and  initiate  social  interactions. 

Team  Orientation 

High  scoring  individuals  prefer  working  in  teams  and  make  people  work 
together  better. 

Virtue 

High  scoring  individuals  strive  to  adhere  to  standards  of  honesty,  morality, 
and  “good  Samaritan”  behavior. 

As  described  further  in  Chapter  7,  these  versions  of  the  TAPAS  will  soon  be  replaced  as  well. 

As  a  test  security  measure,  fonn  equivalence  information  is  provided  in  a  limited  distribution 
addendum.  Scores  have  been  standardized  within  TAPAS  versions  to  enable  cross-version 
analyses.  Descriptive  statistics  and  intercorrelations  of  individual  TAPAS  scale  scores  and 
composite  scores  are  provided  in  Appendix  A. 

TAPAS  Composites 

An  initial  Education  Tier  1  performance  screen  was  developed  from  the  TAPAS-95s  scales  for 
the  purpose  of  testing  in  an  applicant  setting  (Allen  et  ah,  20 10). 6  This  was  accomplished  by  (a) 
identifying  key  criteria  of  most  interest  to  the  Army,  (b)  sorting  these  criteria  into  “can-do”  and 
“will-do”  categories  (see  below),  and  (c)  selecting  composite  scales  corresponding  to  the  can-do 
and  will-do  criteria,  taking  into  account  both  theoretical  rationale  and  empirical  results.  The 
result  of  this  process  was  two  composite  scores. 

Can-Do  Composite:  The  original  TOPS  Operational  Can-Do  composite  consists  of  five  TAPAS 
scales  and  is  designed  to  predict  the  extent  to  which  Soldiers  can  perform  the  technical  aspects  of 
their  jobs,  using  indicators  such  as  MOS-specific  job  knowledge,  Advanced  Individual  Training 
(AIT)  exam  grades,  and  graduation  from  AIT/One  Station  Unit  Training  (OSUT). 

Will-Do  Composite:  The  original  TOPS  Operational  Will-Do  composite  consists  of  five  TAPAS 
scales  (three  of  which  overlap  with  the  Can-Do  composite)  and  is  designed  to  predict  the  more 
motivational  elements  of  job  perfonnance,  such  as  maintaining  physical  fitness,  adjusting  to 
Army  life,  demonstrating  effort,  and  supporting  peers. 

As  more  data  became  available  for  the  dimensions  included  in  the  different  TAPAS  versions, 
additional  work  was  done  to  create  and  evaluate  new  TAPAS  composites.  As  a  result  of  this 
work,  the  Anny  has  approved  the  use  of  three  new  composites  to  screen  applicants.  In  addition  to 

6  TAPAS-95s  was  a  paper-and-pencil,  static  version  of  the  TAPAS  used  in  the  Army  Class  research. 
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reconfigured  Can-Do  and  Will-Do  composites,  there  is  a  “Adaptation”  composite  designed  to 
predict  attrition.  These  new  composites  will  be  used  starting  in  FY2014,  along  with  the 
introduction  of  three  new  versions  of  TAP  AS.  More  infonnation  about  how  the  new  composites 
were  developed  is  provided  in  a  limited  distribution  addendum  in  a  prior  report.  Those  interested 
in  obtaining  a  copy  of  this  addendum  should  contact  the  editors  for  further  infonnation. 

Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 

Content,  Structure,  and  Scoring 

The  ASVAB  is  a  multiple  aptitude  battery  of  tests  administered  by  the  MEPCOM.  Most  military 
applicants  take  the  computer  adaptive  version  of  ASVAB  (i.e.,  the  CAT-ASVAB).  Scores  on  the 
ASVAB  tests  are  combined  to  create  composite  scores  for  use  in  (a)  selecting  applicants  into  the  Anny 
and  (b)  classifying  them  into  an  MOS.  The  AFQT,  the  composite  used  for  selecting  applicants  into  the 
Anny,  comprises  the  Verbal  Expression7 8  (VE),  Arithmetic  Reasoning  (AR),  and  Math  Knowledge 
(MK)  tests  (AFQT  =  2*VE  +  AR  +  MK).  Applicants  must  meet  a  minimum  AFQT  score  to  be  eligible 
to  serve  in  the  military,  and  the  Services  favor  high-scoring  applicants  for  enlistment.  AFQT  percentile 

o 

scores  are  divided  into  the  following  categories: 

•  Category  I  (93-99) 

•  Category  II  (65-92) 

•  Category  IIIA  (50-64) 

•  Category  IIIB  (31-49) 

•  Category  IV  (10-30) 

•  Category  V  (1-9) 

AFQT  Category  V  Soldiers  are  not  eligible  for  enlistment,  Category  IV  accessions  are  greatly 
restricted,  and  priority  is  given  to  Category  I-IIIA  accessions. 

For  classification,  scores  on  the  ASVAB  tests  are  combined  to  form  10  Aptitude  Area  (AA) 
composites.  An  applicant  must  receive  a  minimum  score  on  the  MOS-relevant  AA  composite(s)  to 
qualify  for  classification  to  that  MOS.  For  example,  applicants  must  score  a  95  in  both  the 
Electronics  (EL)  and  Signal  Communications  (SC)  AA  composites  to  qualify  as  a  Signal  Support 
Specialist  (25U).  Descriptive  statistics  for  the  AFQT,  ASVAB  tests,  and  AA  composites  are  reported 
in  Appendix  A.  AFQT  Category  frequencies  are  reported  in  Chapter  2  (Tables  2.1  and  2.2). 

Summary 

The  purpose  of  this  chapter  was  to  describe  the  primary  predictor  measures  used  as  part  of  the 
TOPS  IOT&E.  The  TAPAS  is  unique  among  personality  measures  because  it  uses  forced-choice 
pairwise  items  and  IRT  to  promote  resistance  to  faking.  Initial  validation  research  conducted  as 
part  of  EEEM  was  promising  enough  to  warrant  an  IOT&E  of  the  TAPAS.  The  ASVAB,  which 
consists  of  multiple  tests  that  are  formed  into  operational  selection  (i.e.,  AFQT)  and 
classification  (i.e.,  AA)  composites,  is  used  as  the  baseline  instrument  for  incremental  validity 
analyses  reported  in  Chapters  5  and  6. 

7  Verbal  Expression  is  a  scaled  combination  of  the  Word  Knowledge  (WK)  and  Paragraph  Comprehension  (PC)  tests. 

8  For  more  information  on  ASVAB  scoring,  see  the  official  website  of  the  ASVAB,  www.officialasvab.com. 
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CHAPTER  4:  DESCRIPTION  AND  PSYCHOMETRIC  PROPERTIES  OF  CRITERION 

MEASURES 

Bethany  H.  Bynum  and  Adam  S.  Beatty  (HumRRO) 


Criterion  scores  to  validate  the  TAPAS  were  derived  from  measures  administered  for  purposes 
of  this  research  and  from  administrative  records.  The  research  measures  included  job  knowledge 
tests  (JKTs),  performance  rating  scales  (PRS),  and  a  questionnaire  measuring  self-reported 
attitudes  and  perfonnance  (Anny  Life  Questionnaire  [ALQ]).  The  original  versions  of  these 
three  measures  were  developed  for  the  Anny  Class  project  (Moriarty,  Campbell,  Heffner,  & 
Knapp,  2009),  and  modified,  as  needed,  for  inclusion  in  the  TOPS  IOT&E.  Criterion  scores 
drawn  from  Soldiers’  administrative  records  included  separation  status  (i.e.,  attrition),  IMT 
completion,  and  IMT  grades. 


Table  4.1.  Summary  of  IMT  and  In-Unit  Criterion  Measures 


Criterion  Measure 

Description 

Soldier/Cadre  Reported 

Job  Knowledge  Tests  (JKT) 

The  Warrior  Tasks  and  Battle  Drills  (WTBD)  JKT  measures  knowledge 
that  is  general  to  all  enlisted  Soldiers.  MOS-specific  JKTs  measure 

Soldiers’  knowledge  of  basic  facts,  principles,  and  procedures  required  of 
Soldiers  in  training  for  a  particular  MOS.  Each  JKT  includes  a  mix  of  item 
formats  (e.g.,  multiple-choice,  multiple-response,  and  rank  order). 

Performance  Rating  Scales  (PRS) 

The  IMT  PRS  measure  Soldiers’  performance  in  two  domains:  (a)  MOS- 
specific  (e.g.,  learns  preventive  maintenance  checks  and  services,  learns  to 
troubleshoot  vehicle  and  equipment  problems)  and  (b)  Army-wide  (e.g., 
exhibits  effort,  supports  peers,  demonstrates  physical  fitness).  The  IMT 

PRS  are  completed  by  drill  sergeants  or  training  cadre.  In-unit  PRS  cover 
Army-wide  dimensions  only  and  are  completed  by  supervisors. 

Army  Life  Questionnaire  (ALQ) 

The  ALQ  measures  Soldiers’  self-reported  attitudes  and  experiences  in  the 
Army.  The  IMT  and  in-unit  versions  are  very  similar. 

Administrative 

Attrition 

Separation  data  were  obtained  on  participating  Regular  Army  Soldiers  at  3 
months  (attrition  near  or  after  the  completion  of  Basic  Combat  Training), 

4  months  (attrition  during  AIT  (Advanced  Individual  Training)  /OSUT 
(One  Site  Unit  Training)),  6  months  (attrition  near  or  after  completion  of 
AIT/OSUT),  and  at  regular  3-month  intervals  thereafter.  Attrition  data 
through  30  months  were  available  for  the  current  sample. 

Initial  Military  Training  (IMT) 
Criteria 

These  data  provide  information  about  whether  Soldiers  restarted  IMT  and 
for  what  reasons,  the  number  of  times  Soldiers  restarted  training, 
graduation  status,  and  school  grades  for  Soldiers  in  AIT. 

In  this  chapter,  we  first  describe  the  criterion  measures,  along  with  their  distributional  and 
psychometric  properties.  Then,  we  discuss  the  development  of  a  set  of  criterion  composites, 
providing  distributional  and  psychometric  infonnation  for  these  variables  as  well.  The 
descriptive  statistics  and  correlations  among  the  criterion  composites  (shown  in  Appendix  B)  are 
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based  on  the  Validation  Sample  (i.e.,  Education  Tier  1  and  2,  non-prior  service,  AFQT  Category 
IV  or  above  Soldiers  with  matching  criterion  data).  Descriptive  statistics  and  psychometric 
properties  of  the  criterion  measures  for  the  full  IMT  and  in-unit  samples  are  reported  in 
Appendix  C. 


Job  Knowledge  Tests  (JKTs) 

Multiple  sets  of  JKTs  (IMT  and  in-unit)  were  developed  or  adapted  from  the  Select21  (Collins, 
Le,  &  Schantz,  2005)  and  Army  Class  (Moriarty  et  al.,  2009)  projects:  one  for  Warrior  Tasks  and 
Battle  Drills  (WTBD),  which  is  administered  to  all  participating  Soldiers,  and  a  set  of  MOS- 
specific  JKTs  for  Infantry,  Annor,  Military  Police,  Health  Care  Specialist,  Light  Wheel  Vehicle 
Mechanic,  and  Motor  Transport  Operator  Soldiers.  MOS-specific  JKTs  for  two  additional  MOS, 
Signal  Support  Specialist  and  Human  Resources  Specialist,  were  developed  in  the  Fall  of  201 1. 
Most  of  the  JKT  items  are  in  a  multiple-choice  format  with  two  to  four  response  options. 
However,  other  formats,  such  as  multiple-response  (i.e.,  check  all  that  apply),  rank  ordering,  and 
matching  are  also  used.  The  items  use  visual  images  to  make  them  more  realistic  and  reduce 
reading  requirements  for  the  test. 

A  single,  overall  raw  score  was  computed  for  each  JKT  by  summing  the  total  number  of  points 
Soldiers  earned  across  the  JKT  items  and  computing  a  percent  correct  score  based  on  the 
maximum  number  of  points  that  could  be  obtained  on  each  test.  For  the  criterion-related  validity 
analyses,  we  converted  the  total  raw  score  to  a  standardized  score  (or  z-score)  by  standardizing 
the  scores  within  each  MOS.  JKT  scores  were  flagged  as  not  usable  if  the  Soldier  (a)  omitted 
more  than  10%  of  the  assessment  items,  (b)  took  fewer  than  5  minutes  to  complete  the  entire 
assessment,  or  (c)  selected  an  implausible  response  to  one  of  the  careless  responding  items 
(Knapp  et  al.,  2012).  Table  4.2  lists  the  reliability  estimates  for  the  MOS-Specific  JKTs  and  the 
Army-wide  JKT  for  the  IMT  and  in-unit  sample. 


Table  4.2.  Reliability  Estimates  of  the  Job  Knowledge  Tests  (JKTs)  in  the  IMT  and  In-Unit 
Validation  Samples 


n 

a 

n 

a 

MOS-Speciftc  (IMT) 

MOS-Speciftc  (In-Unit) 

11B/C/X  +  18X 

5,383 

.77 

11B/C/X  +  18X 

161 

.70 

19K 

302 

.75 

19K 

15 

— 

31B 

2,538 

.75 

31B 

29 

— 

42  A 

245 

.75 

42  A 

29 

- 

68W 

2,705 

.89 

68W 

35 

— 

88M 

2,080 

.76 

88M 

52 

.77 

91B 

262 

.90 

91B 

48 

- 

WTBD  (Army-Wide) 

16,086 

.67 

WTBD  (Army-Wide) 

1,019 

.54 

Note.  WTBD  =  Warrior  Tasks  and  Battle  Drills,  a  =  coefficient  alpha.  Statistics  based  on  fewer  than  50  cases  are  not  reported. 
Reported  alpha  is  the  composite  reliability  of  the  scales  encompassing  the  criterion  composite.  Reliability  estimates  across  MOS 
were  averaged  to  represent  the  All  MOS  combined  MOS-Specific  JKT  reliability. 


Table  4.3  lists  the  descriptive  statistics  for  the  IMT  Army- wide  and  MOS-specific  JKTs  by 
education  tier  and  Table  4.4  lists  the  descriptive  statistics  for  the  in-unit  Army-wide  and  MOS- 
specific  JKTs  by  education  tier. 


16 


Table  4.3.  Descriptive  Statistics  for  the  Job  Knowledge  Tests  (JKTs)  by  Education  Tier  in  the 
IMT  Validation  Sample 


Domain/JKT 

n 

M 

SD 

Min 

Max 

r  WTBD 

rAFQT 

Tier  1 

MOS-Specific 

11B/C/X  +  18X 

5,153 

60.90 

10.37 

25.58 

86.05 

.57 

.45 

19K 

284 

61.64 

12.05 

20.29 

85.71 

.44 

.26 

31B 

2,471 

67.70 

8.25 

35.96 

91.26 

.49 

.47 

42  A 

234 

55.78 

12.38 

25.93 

81.48 

.54 

.50 

68W 

2,619 

72.74 

10.66 

26.00 

92.39 

.50 

.26 

88M 

1,997 

63.61 

9.98 

31.94 

88.89 

.55 

.40 

91B 

253 

57.21 

13.61 

27.84 

90.72 

.56 

.27 

All  MOS  Combined 

13,011 

64.84 

11.24 

20.29 

92.39 

.56 

.46 

WTBD  (Army- Wide) 

15,476 

64.60 

12.56 

6.45 

96.77 

— 

.43 

Tier  2 

MOS-Specific 

11B/C/X  +  18X 

230 

60.14 

10.95 

26.09 

86.96 

.58 

.24 

31B 

67 

69.51 

7.53 

52.43 

85.44 

.41 

.36 

68W 

86 

70.92 

10.46 

33.70 

84.00 

.57 

.18 

88M 

83 

64.68 

10.29 

34.72 

83.33 

.62 

.47 

All  MOS  Combined  a 

504 

63.76 

11.46 

26.09 

86.96 

.57 

.31 

WTBD  (Army-Wide) 

610 

65.09 

11.87 

16.13 

93.55 

— 

.28 

Tier  1  +  2  (Combined) 

MOS-Specific 

11B/C/X  +  18X 

5,383 

60.86 

10.39 

25.58 

86.96 

.57 

.44 

19K 

302 

61.43 

11.98 

20.29 

85.71 

.44 

.25 

31B 

2,538 

67.74 

8.23 

35.96 

91.26 

.49 

.47 

42  A 

245 

55.85 

12.28 

25.93 

81.48 

.53 

.49 

68W 

2,705 

72.68 

10.66 

26.00 

92.39 

.50 

.26 

88M 

2,080 

63.66 

10.00 

31.94 

88.89 

.55 

.41 

91B 

262 

57.15 

13.74 

27.84 

90.72 

.56 

.24 

All  MOS  Combined a 

13,515 

64.80 

11.25 

20.29 

92.39 

.56 

.46 

WTBD  (Army-Wide) 

16,086 

64.62 

12.54 

6.45 

96.77 

- 

.43 

Note.  Ms,  SDs ,  Min,  and  Max  reflect  percent  correct,  a  =  coefficient  alpha.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  rWTBD  = 
correlation  with  WTBD  JKT  scores.  /*afqt  =  correlation  with  AFQT  scores.  All  correlations  are  statistically  significant  (p  <  .05, 
one-tailed). Statistics  based  on  fewer  than  50  cases  are  not  separately  reported. 
a  Includes  1  IB,  19K,  3 IB,  42A,  68W,  88M,  and  9 IB. 
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Table  4.4.  Descriptive  Statistics  for  the  Job  Knowledge  Tests  (JKTs)  by  Education  Tier  in  the 
In-Unit  Validation  Sample 


Domain/JKT 

n 

M 

SD 

Min 

Max 

r  WTBD 

rAFQT 

Tier  1 

MOS-Specific 

11B/C/X  +  18X 

159 

62.31 

9.96 

26.76 

83.10 

.60 

.49 

88M 

51 

68.05 

8.74 

44.68 

85.15 

.67 

.62 

All  MOS  Combined  a 

360 

64.43 

10.98 

26.76 

90.20 

.50 

.47 

WTBD  (Army  Wide) 

998 

65.61 

11.74 

23.08 

96.15 

— 

.46 

Tier  1  +  2  (Combined)*5 

MOS-Specific 

11B/C/X  +  18X 

161 

62.24 

9.93 

26.76 

83.10 

.60 

.49 

88M 

52 

67.79 

8.85 

44.68 

85.15 

.69 

.64 

All  MOS  Combined a 

369 

64.45 

11.01 

26.76 

90.20 

.50 

.47 

WTBD  (Army  Wide) 

1,019 

65.54 

11.69 

23.08 

96.15 

— 

.46 

Note.  Ms,  SDs ,  Min,  and  Max  reflect  percent  correct,  a  =  coefficient  alpha.  WTBD  =  Warrior  Tasks  and  Battle  Drills,  rWTBD  = 
correlation  with  WTBD  JKT  scores.  ?afqt  =  correlation  with  AFQT  scores.  All  correlations  are  statistically  significant  (p  <  .05, 
one-tailed). Statistics  based  on  fewer  than  50  cases  are  not  separately  reported. 

“  Includes  1  IB,  19K,  3  IB,  42A,  68W,  88M,  and  9 IB. 

b  Statistics  for  Tier  2  Soldiers  are  not  reported  separately  because  of  insufficient  sample  size  (n  <  10). 


Performance  Rating  Scales  (PRS) 

The  PRS,  like  the  JKTs,  also  were  adapted  from  or  based  on  previous  research  (see  Moriarty  et 
ah,  2009  for  details).  The  IMT  and  in-unit  PRS  are  fairly  different,  so  they  will  be  described 
separately. 


IMT  PRS 

The  IMT  PRS  target  two  domains  of  Soldier  performance  requirements:  (a)  Army-wide  and  (b) 
MOS-specific.  The  IMT  PRS  were  completed  by  cadre  members  (drill  sergeants,  trainers)  of 
participating  Soldiers. 

Over  the  course  of  the  TOPS  IOT&E,  two  versions  of  the  IMT  PRS  were  administered.  Early 
IOT&E  evaluations  noted  low  interrater  reliability  estimates  for  the  PRS  (Moriarty  &  Bynum,  2011). 
Accordingly,  several  changes  were  made  to  the  IMT  instruments  in  an  attempt  to  improve  their 
psychometric  characteristics.  First,  the  number  of  scales  for  the  Anny-wide  PRS  was  reduced  from 
eight  to  five,  paralleling  the  five  scores  generated  from  the  original  scales  (Sparks  &  Peddie,  2013). 
No  changes  were  made  to  the  MOS-specific  PRS  scales;  the  number  of  scales  ranged  from  five  to 
nine  and  a  composite  score  was  computed  by  averaging  ratings  across  the  individual  scales  for 
each  MOS.  Second,  the  rating  scale  for  both  the  Anny-wide  and  MOS-specific  PRS  was  changed 
from  a  7-point  Behaviorally  Anchored  Rating  Scale  (BARS)  to  a  5-point  relative  scale  fonnat  with 
scales  ranging  from  1  (Among  the  Weakest)  to  5  (Among  the  Best).  Ratings  on  the  initial  PRS 
rating  scale  were  re-scaled  to  reflect  the  new  5-point  scale.  After  doing  so,  we  combined  ratings 
data  across  the  two  versions  of  PRS  to  create  a  single  PRS  score.  All  IMT  PRS  results  reported 
are  based  on  data  from  the  initial  and  revised  PRS,  and  are  expressed  on  a  5 -point  scale.  Finally, 
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cadre  members  also  indicated  their  opportunity  to  observe  each  Soldier  being  rated  using  a  4- 
point  “familiarity”  scale,  in  which  the  cadre  rated  his  or  her  opportunity  to  observe  each  Soldier 
being  rated.  The  initial  PRS  used  a  3-point  familiarity  scale.  This  was  changed  to  a  4-point  scale 
to  enable  raters  to  more  clearly  indicate  their  ability  to  judge  each  Soldier’s  perfonnance. 

Table  4.5  compares  the  estimated  interrater  reliability  for  the  initial  and  revised  versions  of  the 
IMT  PRS.  The  interrater  reliability  estimates  were  generally  low  (.30  or  less)  for  the  initial 
version  of  the  PRS.  As  hoped,  the  revised  PRS  showed  increased  interrater  reliability  with  many 
of  the  reliability  estimates  more  than  doubling.  However,  generally  the  estimates  were  still  low. 
We  attribute  these  low  coefficients  to  several  interrelated  issues.  First,  the  number  of  ratees  per 
rater  is  high,  averaging  about  16  with  some  raters  providing  as  many  as  50  ratings.  As  a  result, 
raters  likely  became  fatigued  during  the  rating  task.  Second,  within-rater  variance  was  generally 
limited,  perhaps  reflecting  raters’  inability  to  differentiate  among  individual  Soldiers.  Third, 
these  data  collections  were  not  proctored,  unlike  prior  research  (e.g.,  Knapp  &  Heffner,  2009, 
2010).  Finally,  the  number  of  raters  per  ratee  was  small,  averaging  less  than  two,  which  limits 
the  generalizability  of  single-rater  reliability  estimates. 

Table  4.5.  Interrater  Reliability  Estimates  for  the  Performance  Rating  Scales  (PRS)  in  the 
IMT  Validation  Sample 


Original  PRS 

Revised  PRS 

Anny-  Wide  PRS 

n 

IRR 

n 

IRR 

Commitment  &  Adjustment 

12,256 

.139 

2,635 

.310 

Effort  &  Personal  Discipline 

12,283 

.169 

2,636 

.423 

MOS  Qualification  Knowledge 

11,100 

.095 

2,324 

.250 

Physical  Fitness  &  Bearing 

12,184 

.191 

2,634 

.235 

Working  with  Others 

12,216 

.150 

2,631 

.319 

Overall  Performance 

12,087 

.318 

2,593 

.350 

MOS-Specific  PRS 

11B/C/X+  18X 

3,981 

.174 

571 

.208 

19K 

158 

.405 

- 

- 

31B 

1,798 

.107 

554 

.435 

68W 

3,099 

.009 

181 

.074 

88M 

672 

.000 

- 

- 

91B 

255 

.106 

- 

- 

Note,  a  =  coefficient  alpha.  IRR  =  Interrater  reliability.  Interrater  reliability  was  assessed  using  G(q,k),  a  reliability  metric 
designed  specifically  for  studies  like  TOPS  where  the  measurement  design  is  ill-structured  (Putka,  Le,  McCloy,  &  Diaz,  2008). 
IRR  estimates  were  not  estimated  if  30  or  fewer  Soldiers  were  rated  by  more  than  one  supervisor.  Approximately  20%  of  Soldiers 
with  IMT  PRS  data  were  rated  by  more  than  one  cadre  member. 


Table  4.6  summarizes  the  descriptive  statistics  for  IMT  PRS  by  education  tier.  A  Soldier’s  PRS 
ratings  were  not  included  in  the  analyses  if  the  rater  (a)  indicated  he  or  she  had  little  opportunity  to 
observe  this  Soldier,  (b)  omitted  more  than  10%  of  the  assessment  items,  (c)  indicated  that  he  or  she 
had  not  observed  the  Soldier  on  more  than  50%  of  the  dimensions,  or  (d)  engaged  in  “flat 
responding” — that  is,  if  the  rater  rated  10  or  more  Soldiers  on  a  particular  scale  and  90%  or  more  of 
those  rating  profiles  were  exactly  the  same.  Mean  ratings  were  above  the  mid-point,  a  consistent 
finding  in  prior  Anny  research  involving  perfonnance  ratings  (e.g.,  Campbell  &  Knapp,  2001; 

Knapp  &  Tremble,  2007;  Moriarty  &  Bynum,  2011).  The  IMT  PRS  were  also  highly  interconelated 
(see  Appendix  B). 
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Table  4.6.  Descriptive  Statistics  for  the  Performance  Rating  Scales  ( PRS )  by  Education  Tier  in 


the  IMT  Validation  Sample 


Domain/Setting/PRS 

n 

M 

SD 

Min 

Max 

Tier  1 

Army-  Wide 

Adjustment  to  the  Army 

5,160 

3.40 

1.00 

1.00 

5.00 

Effort  &  Personal  Discipline 

5,165 

3.22 

.98 

1.00 

5.00 

MOS  Qualification  Knowledge 

4,835 

3.33 

.93 

1.00 

5.00 

Physical  Fitness  &  Bearing 

5,149 

3.28 

.99 

1.00 

5.00 

Working  with  Others 

5,152 

3.21 

.98 

1.00 

5.00 

Overall  Performance 

5,123 

3.56 

.83 

1.00 

5.00 

MOS-Specific 

11B/C/X+  18X 

1,462 

3.18 

.84 

1.00 

5.00 

19K 

79 

3.31 

.62 

1.71 

4.86 

31B 

1,037 

3.31 

.75 

1.12 

5.00 

68W 

764 

2.98 

.86 

1.00 

5.00 

88M 

115 

2.87 

.78 

1.20 

5.00 

All  MOS  Combined 

3,500 

3.16 

.82 

1.00 

5.00 

Tier  2 

Army-  Wide 

Adjustment  to  the  Army 

111 

3.45 

.91 

1.00 

5.00 

Effort  &  Personal  Discipline 

111 

3.28 

.88 

1.00 

5.00 

MOS  Qualification  Knowledge 

171 

3.45 

.80 

2.00 

5.00 

Physical  Fitness  &  Bearing 

111 

3.27 

.86 

1.00 

5.00 

Working  with  Others 

111 

3.39 

.95 

1.00 

5.00 

Overall  Performance 

176 

3.45 

.83 

1.00 

5.00 

MOS-Specific 

11B/C/X  +  18X 

56 

3.18 

.71 

1.83 

5.00 

All  MOS  Combined 

119 

3.14 

.73 

1.00 

5.00 

Tier  1  +  Tier  2  (Combined) 

Army-  Wide 

Commitment  &  Adjustment 

5,337 

3.40 

.99 

1.00 

5.00 

Effort  &  Personal  Discipline 

5,342 

3.22 

.97 

1.00 

5.00 

MOS  Qualification  Knowledge 

5,006 

3.34 

.92 

1.00 

5.00 

Physical  Fitness  &  Bearing 

5,326 

3.28 

.99 

1.00 

5.00 

Working  with  Others 

5,329 

3.21 

.98 

1.00 

5.00 

Overall  Performance 

5,299 

3.56 

.83 

1.00 

5.00 

MOS-Specific 

11B/C/X+  18X 

1,518 

3.18 

.83 

1.00 

5.00 

19K 

86 

3.32 

.62 

1.71 

4.86 

31B 

1,064 

3.31 

.74 

1.12 

5.00 

68W 

789 

2.98 

.86 

1.00 

5.00 

88M 

119 

2.86 

.78 

1.20 

5.00 

All  MOS  Combined 

3,619 

3.16 

.82 

1.00 

5.00 

Note.  Statistics  based  on  fewer  than  50  cases  are  not  reported.  Ratings  on  PRS  range  from  1  and  5.  PRS  ratings  from  supervisors 


with  a  familiarity  rating  of  1  (“I  have  had  little  opportunity  to  observe  this  Soldier”)  were  excluded  from  analyses. 
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In-Unit  PRS 


The  in-unit  PRS  only  target  Army-wide  dimensions  of  performance  (i.e.,  there  are  no  MOS- 
specific  in-unit  PRS)  and  include  12  perfonnance  dimensions,  plus  a  Leadership  Potential  scale 
(see  Table  4.10).  A  thirteenth  scale  was  dropped  in  Fall  2011  because  of  poor  psychometric 
properties  and  has  since  been  replaced  with  an  Adjustment  scale,  comparable  to  the 
corresponding  IMT  scale.  Ratings  on  several  of  the  individual  scales  are  combined  to  fonn  four 
PRS  composites  and  three  scales  were  left  as  standalone  dimensions.  Cronbach’s  alpha 
coefficients  for  the  in-unit  PRS  composite  scales  are  reported  in  Table  4.7.  The  in-unit  PRS  have 
consistently  employed  the  7-point  BARS  fonnat  used  for  the  initial  IMT  scales.  The  revised  4- 
point  “familiarity”  scale  used  in  the  new  IMT  PRS  also  is  used  with  the  in-unit  PRS.  The 
majority  of  Soldiers  in  units  were  rated  by  only  one  supervisor,  so  interrater  reliability  estimates 
were  not  calculated.  Table  4.8  reports  the  basic  descriptive  statistics  for  the  in-unit  Army-wide 
PRS  by  perfonnance  domain  and  education  tier. 

Table  4.7.  In-Unit  Army-Wide  Performance  Rating  Scale  Dimensions  and  Composite  Scores 


In-Unit  Rating  Composites  and  Dimensions 

a 

Can  Do 

Performing  Core  Warrior  Tasks 

Performing  MOS-Speciftc  Tasks 

Processing  Information 

Solving  Problems 

.84 

Effort  &  Personal  Discipline 

Exhibiting  Effort 

Exhibiting  Personal  Discipline 

.80 

Working  with  Others 

Communicating  with  Others 

Contributing  to  the  Team 

.77 

Self-Management 

Following  Safety  Procedures 

Developing  Own  Skills 

Managing  Personal  Matters 

Adjusting  to  Army  Life 

Physical  Fitness  and  Bearing 

Overall  Leadership  Potential  Rating 

.79 

Note.  Of  the  seven  scores  computed,  four  are  composites  of  multiple  dimensions  and  three  are  single  dimension  ratings. 
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Table  4.8.  Descriptive  Statistics  and  Reliability  Estimates  for  the  Performance  Rating  Scales 
(PRS)  in  the  In-Unit  Validation  Sample 


PRS  Dimensions/Composites 

n 

M 

SD 

Min 

Max 

Tier  1 

Can  Do  a 

719 

4.90 

1.27 

1.00 

7.00 

Effort  &  Personal  Discipline  a 

719 

5.29 

1.34 

1.00 

7.00 

Working  with  Others  a 

719 

5.32 

1.24 

1.50 

7.00 

Self-Management a 

718 

5.31 

1.13 

1.33 

7.00 

Physical  Fitness  &  Bearing 

715 

5.36 

1.56 

1.00 

7.00 

Overall  Leadership  Potential 

697 

4.88 

1.66 

1.00 

7.00 

Tier  1  +  Tier  2  (Combined)*5 

Can  Do  a 

734 

4.88 

1.27 

1.00 

7.00 

Effort  &  Personal  Discipline  a 

734 

5.27 

1.34 

1.00 

7.00 

Physical  Fitness  &  Bearing 

730 

5.34 

1.56 

1.00 

7.00 

Self-Management a 

733 

5.30 

1.13 

1.33 

7.00 

Working  with  Others  a 

734 

5.31 

1.25 

1.50 

7.00 

Overall  Leadership  Potential 

711 

4.86 

1.66 

1.00 

7.00 

Note.  Ratings  on  PRS  range  from  1  and  7.  PRS  ratings  from  supervisors  with  a  familiarity  rating  of  1  (“I  have  had  little 
opportunity  to  observe  this  Soldier”)  were  excluded  from  analyses. 
a  Ratings  composite  comprises  two  or  more  Army-wide  PRS. 

b  Statistics  for  Tier  2  Soldiers  are  not  reported  separately  since  the  sample  size  is  less  than  50  for  all  PRS  scores. 


Army  Life  Questionnaire  (ALQ) 

The  ALQ  was  designed  to  measure  Soldiers’  self-reported  attitudes  and  experiences  in  the 
Army.  Earlier  forms  of  the  training  and  in-unit  versions  of  the  ALQ  (Van  Iddekinge,  Putka,  & 
Sager,  2005)  were  modified  slightly  for  use  in  the  TOPS  IOT&E.  The  ALQ  includes  scales 
that  cover  (a)  Soldiers’  commitment  and  retention-related  attitudes  and  (b)  Soldiers’ 
performance  and  adjustment.  Each  ALQ  scale  is  scored  differently  depending  on  the  nature  of 
the  attribute  being  measured.  The  Army  Physical  Fitness  Test  (APFT)  score  is  a  write-in  item. 
Training  Achievements,  Training  Restarts,  (both  of  which  appear  only  on  the  IMT  version  of 
the  ALQ),  and  Disciplinary  Incidents  are  simply  a  sum  of  the  “yes”  responses.  The  remaining 
scales  (see  Table  4.9)  are  composed  of  Likert-type  response  scales  and  are  scored  by 
computing  a  mean  of  the  constituent  item  scores.  Note  that  most  scales  appear  on  both  the  IMT 
and  in-unit  versions  of  the  scales,  though  the  IMT  version  has  two  unique  scales  (i.e., 
Normative  Commitment  and  Army  Life  Adjustment). 
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Table  4.9.  Army  Life  Questionnaire  (ALQ)  Likert-Type  Scales 


Scale  Name 

Description 

Number 
of  Items 

Example  Item 

Likert  Scale  Anchors 

Affective 

Commitment 

Measures  Soldiers’ 
emotional  attachment  to 
the  Army. 

7 

I  feel  like  I  am  part  of  the 
Army  ‘family.’ 

1  (strongly  disagree)  to  5 
(strongly  agree) 

Normative 
Commitment a 

Measures  Soldiers’  feelings 
of  obligation  toward 
staying  in  the  Army  until 
the  end  of  their  current 
term  of  service. 

5 

I  would  feel  guilty  if  I  left  the 
Army  before  the  end  of  my 
current  term  of  service. 

1  (strongly  disagree)  to  5 
(strongly  agree) 

Career 

Intentions 

Measures  Soldiers’ 
intentions  to  reenlist  and  to 
make  the  Army  a  career. 

3 

How  likely  is  it  that  you  will 
make  the  Army  a  career? 

Varies  by  item:  1  (strongly 
disagree)  to  5  (strongly 
agree);  1  (not  at  all 
confident)  to  5  (extremely 
confident);  1  (extremely 
unlikely)  to  5  (extremely 
likely) 

Reenlistment 

Intentions 

Measures  Soldiers’ 
intention  to  reenlist  in  the 
Army. 

4 

How  likely  is  it  that  you  will 
leave  the  Army  after 
completing  your  current  term 
of  service? 

1  (strongly  disagree)  to  5 
(strongly  agree) 

Attrition 

Cognitions 

Measures  the  degree  to 
which  Soldiers  think  about 
attriting  before  the  end  of 
their  first  term. 

4 

How  likely  is  it  that  you  will 
complete  your  current  term  of 
service? 

Varies  by  item:  1  (strongly 
disagree)  to  5  (strongly 
agree);  1  (never)  to  5  (very 
often) 

Army  Life 
Adjustment a 

Measures  Soldiers’ 
transition  from  civilian  to 
Army  life. 

9 

Looking  back,  I  was  not 
prepared  for  the  challenges  of 
training  in  the  Army. 

1  (strongly  disagree)  to  5 
(strongly  agree) 

MOS  Fit 

Measures  Soldiers’ 
perceived  fit  with  their 
MOS. 

9 

My  MOS  provides  the  right 
amount  of  challenge  for  me. 

1  (strongly  disagree)  to  5 
(strongly  agree) 

Army  Fit  b 

Measures  Soldiers’ 
perceived  fit  with  the 

Army. 

8 

The  Army  is  a  good  match 
for  me. 

1  (strongly  disagree)  to  5 
(strongly  agree) 

aAppears  only  on  the  IMT  ALQ. 
b  Scale  has  6  items  on  the  in-unit  ALQ. 


ALQ  data  were  flagged  as  unusable  if  the  Soldier  (a)  omitted  more  than  10%  of  the  assessment 
items,  (b)  took  fewer  than  5  minutes  to  complete  the  entire  assessment,  or  (c)  chose  an 
implausible  response  to  the  careless  responding  item.  Table  4.10  and  Table  4.1 1  summarize  the 
descriptive  statistics  for  the  ALQ  scales  by  education  tier  for  the  IMT  and  in-unit  samples, 
respectively. 
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Table  4.10.  Descriptive  Statistics  and  Reliability  Estimates  for  the  Army  Life  Questionnaire 


(ALQ)  by  Education  Tier  in  the  IMT  Validation  Sample 


Domain/Scale 

n 

M 

SD 

Min 

Max 

a 

Tier  1 

Retention 

Army  Career  Intentions 

16,201 

3.17 

1.08 

1.00 

5.00 

- 

Affective  Commitment 

16,201 

3.88 

.67 

1.00 

5.00 

- 

Army  Life  Adjustment 

16,201 

4.09 

.66 

1.00 

5.00 

- 

Army  Fit 

16,201 

4.08 

.60 

1.00 

5.00 

- 

Army  Reenlistment  Intentions 

16,201 

3.46 

.95 

1.00 

5.00 

- 

Attrition  Cognition 

16,201 

1.51 

.60 

1.00 

5.00 

- 

MOS  Fit 

16,201 

3.78 

.84 

1.00 

5.00 

- 

Normative  Commitment 

16,201 

4.18 

.69 

1.00 

5.00 

— 

Achievement/Perfonnance 

Disciplinary  Incidents  (#) 

14,582 

.27 

.62 

.00 

7.00 

- 

Last  APFT  Score 

16,041 

252.44 

30.53 

109.00 

300.00 

- 

Training  Achievement  (#) 

16,187 

.41 

.62 

.00 

2.00 

— 

Training  Restarts  (#) 

16,200 

.39 

.63 

.00 

4.00 

- 

Tier  2 

Retention 

Army  Career  Intentions 

634 

3.34 

1.09 

1.00 

5.00 

- 

Affective  Commitment 

634 

4.00 

.67 

1.00 

5.00 

- 

Army  Life  Adjustment 

634 

4.12 

.67 

1.22 

5.00 

- 

Army  Fit 

634 

4.13 

.63 

1.62 

5.00 

- 

Army  Reenlistment  Intentions 

Attrition  Cognition 

634 

1.47 

.58 

1.00 

4.00 

- 

MOS  Fit 

634 

3.85 

.82 

1.11 

5.00 

- 

Normative  Commitment 

634 

4.24 

.65 

2.00 

5.00 

- 

Achievement/Perforamance 

Disciplinary  Incidents  (#) 

600 

.33 

.75 

.00 

6.00 

-- 

Last  APFT  Score 

628 

244.90 

31.96 

124.00 

300.00 

- 

Training  Achievement  (#) 

633 

.37 

.58 

.00 

2.00 

- 

Training  Restarts  (#) 

634 

.37 

.60 

.00 

3.00 

— 

Tier  1  +  Tier  2  (Combined) 

Retention 

Army  Career  Intentions 

16,835 

3.17 

1.08 

1.00 

5.00 

.91 

Affective  Commitment 

16,835 

3.89 

.67 

1.00 

5.00 

.86 

Army  Life  Adjustment 

16,835 

4.09 

.66 

1.00 

5.00 

.87 

Army  Fit 

16,835 

4.08 

.60 

1.00 

5.00 

.86 

Army  Reenlistment  Intentions 

16,835 

3.47 

.95 

1.00 

5.00 

.83 

Attrition  Cognition 

16,835 

1.51 

.59 

1.00 

5.00 

.76 

MOS  Fit 

16,835 

3.79 

.84 

1.00 

5.00 

.92 

Normative  Commitment 

16,835 

4.18 

.69 

1.00 

5.00 

.79 

Achievement/Performance 

Disciplinary  Incidents  (#) 

15,182 

.28 

.62 

.00 

7.00 

-- 

Last  APFT  Score 

16,669 

252.15 

30.62 

109.00 

300.00 

- 

Training  Achievement  (#) 

16,820 

.41 

.61 

.00 

2.00 

- 

Training  Restarts  (#) 

16,834 

.39 

.63 

.00 

4.00 

- 

Note,  a  =  coefficient  alpha.  Reliability  estimates  of  the  ALQ  scales  are  reported  for  the  hill  validation  sample  only. 
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Table  4.11.  Descriptive  Statistics  and  Reliability  Estimates  for  the  Army  Life  Questionnaire 
(ALQ)  by  Education  Tier  in  the  In-Unit  Validation  Sample 


Domain/Setting/Scale 

n 

M 

SD 

Min 

Max 

Tier  1 

Retention 

Affective  Commitment 

1,012 

3.62 

.78 

1.00 

5.00 

Army  Career  Intentions 

1,012 

2.69 

1.16 

1.00 

5.00 

Army  Reenlistment  Intentions 

1,011 

3.08 

1.16 

1.00 

5.00 

Attrition  Cognition 

1,012 

1.66 

.72 

1.00 

5.00 

Army  Fit 

1,012 

3.95 

.67 

1.60 

5.00 

MOS  Fit 

1,012 

3.28 

.95 

1.00 

5.00 

MOS  Satisfaction 

1,012 

3.58 

.87 

1.00 

5.00 

Achievement/Performance 

Disciplinary  Incidents  (#) 

1,012 

.31 

.74 

.00 

7.00 

Last  APFT  Score 

988 

246.74 

31.80 

120.00 

300.00 

Tier  1  +  Tier  2  (Combined) a 

Retention 

Affective  Commitment 

1,034 

3.63 

.77 

1.00 

5.00 

Army  Career  Intentions 

1,034 

2.70 

1.17 

1.00 

5.00 

Army  Reenlistment  Intentions 

1,033 

3.09 

1.16 

1.00 

5.00 

Army  Fit 

1,034 

3.96 

.67 

1.60 

5.00 

Attrition  Cognition 

1,034 

1.66 

.72 

1.00 

5.00 

MOS  Fit 

1,034 

3.28 

.95 

1.00 

5.00 

MOS  Satisfaction 

1,034 

3.59 

.87 

1.00 

5.00 

Achievement/Performance 

Disciplinary  Incidents  (#) 

1,034 

.34 

.80 

.00 

7.00 

Last  APFT  Score 

1,009 

246.66 

31.68 

120.00 

300.00 

Note,  a  =  coefficient  alpha. 

aTier  2  estimates  not  reported  since  all  samples  sizes  are  less  than  50. 
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Administrative  Criteria 


Attrition 

Attrition  is  a  broad  category  that  encompasses  involuntary  and  voluntary  separations  for  a 
variety  of  reasons  (e.g.,  underage  enlistment,  conduct,  family  concerns,  drugs  or  alcohol, 
perfonnance,  physical  standards  or  weight,  mental  disorder,  or  violations  of  the  Uniform  Code  of 
Military  Justice  [UCMJ]).  The  reason  for  separation  was  detennined  by  the  Soldiers’  Separation 
Program  Designator  (SPD)  code.  Soldiers  who  were  classified  as  “attrits”  (coded  as  1)  for 
reasons  outside  of  the  Soldiers’  or  the  Anny’s  control  were  excluded  in  our  analyses  (e.g.,  death 
or  serious  injury  incurred  while  perfonning  one's  duties).  Separation  data  were  limited  to 
Regular  Army  Soldiers  because  of  questions  regarding  the  reliability  of  administrative  separation 
data  for  Reserve  and  National  Guard  Soldiers.  The  current  analyses  cover  attrition  through  30 
months  of  service.  Table  4.12  summarizes  the  basic  descriptive  statistics  for  attrition  by 
education  tier. 


AIT  Grade 

Soldiers’  final  AIT  course  grades  were  extracted  from  RITMS  (Resident  Individual  Training 
Management  System).  Final  grades  from  One  Station  Unit  Training  (OSUT)  courses  were 
excluded  from  the  data  file  because  the  variance  in  the  grades  was  highly  restricted  or  based  on  a 
pass-fail  metric  that  was  redundant  with  the  data  from  ATRRS  (Army  Training  Requirements 
and  Resources  System).  Final  AIT  Grade  represents  the  final  cumulative  grade  administratively 
recorded  for  the  Soldier  upon  graduation  from  AIT.  A  standardized  version  of  Final  AIT  Grade 
was  computed  for  those  MOS  graduating  15  or  more  Soldiers.  Table  4.13  summarizes  the  final 
AIT  grade  by  education  tier. 


Training  Restarts 

Soldiers’  IMT  completion  status  and  whether  he  or  she  graduated  from  IMT  with  a  training 
restarts  were  extracted  from  ATRRS.  Soldiers  who  had  not  had  an  opportunity  to  complete  their 
IMT  at  the  time  data  were  extracted  were  excluded  from  our  analyses.  Table  4.13  presents  the 
base  rates  of  Soldiers  with  at  least  one  training  restart  during  IMT. 
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Table  4.12.  Base  Rates  for  Attrition  Criteria  by  Education  Tier  in  the  Validation  Sample 


Domain/Measure 

Tier  1 

Tier  2 

Tier  1  +  Tier  2  (Combined) 

Na 

NAttrit 

%Attrit 

Na 

NAttrit 

0//OAttrit 

Na 

NAttrit 

0//°Attrit 

Cumulative  Attrition 

3 -Month 

73,989 

5,053 

6.8 

2,720 

287 

10.6 

76,709 

5,340 

7.0 

6-Month 

64,000 

6,447 

10.1 

2,061 

316 

15.3 

66,061 

6,763 

10.2 

9-Month 

52,195 

5,909 

11.3 

1,347 

221 

16.4 

53,542 

6,130 

11.4 

12-Month 

44,783 

5,567 

12.4 

920 

165 

17.9 

45,703 

5,732 

12.5 

15 -Month 

38,227 

5,163 

13.5 

407 

79 

19.4 

38,634 

5,242 

13.6 

18 -Month 

30,705 

4,514 

14.7 

222 

44 

19.8 

30,927 

4,558 

14.7 

21 -Month 

21,665 

3,526 

16.3 

82 

18 

22.0 

21,747 

3,544 

16.3 

24-Month 

16,010 

2,821 

17.6 

79 

18 

22.8 

16,089 

2,839 

17.6 

27-Month 

9,743 

1,853 

19.0 

76 

19 

25.0 

9,819 

1,872 

19.1 

30-Month 

4,750 

958 

20.2 

74 

19 

25.7 

4,824 

977 

20.3 

a  N—  number  of  Regular  Army  Soldiers  with  attrition  data  at  the  time  data  were  extracted. 

Nautu  ~  number  of  Soldiers  who  attrited  at  the  specified  months  of  service.  %Amit  =  percentage  of  Soldiers  who  attrited  through  the  specified  months  of  service  [( NAttrit  IN)  x  100]. 


to 
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Table  4.13.  Base  Rates  or  Basic  Descriptive  Statistics  for  Administrative  IMT  Criteria  in  the  Validation  Sample 


Restarted  Initial  Military >  Training  (IMT) 

Tier  1 

Tier  2 

Tier  1  +  Tier  2  (Combined) 

Na 

N Restarted 

N) Restarted 

Na 

N Restarted 

N) Restarted 

Na 

N Restarted 

N) Restarted 

At  Least  Once  During  IMT 

71,323 

9,112 

12.8 

2,749 

470 

17.1 

74,072 

9,582 

12.9 

For  Academic  or  Other  Pejorative  Reasons 

70,318 

8,087 

11.5 

2,714 

434 

16.0 

73,032 

8,521 

11.7 

For  Academic  Reasons 

68,675 

6,460 

9.4 

2,549 

270 

10.6 

71,224 

6,730 

9.4 

Final  AIT  School  Grades 

Nb 

M 

SD 

Nh 

M 

SD 

Nb 

M 

SD 

Overall  Average  (Unstandardized) 

40,568 

91.69 

8.13 

1,641 

91.72 

7.73 

42,209 

91.69 

8.11 

Overall  Average  (Standardized  within  MOS) 

40,276 

.05 

.80 

1,630 

.00 

.80 

41,906 

.05 

.80 

aIV=  number  of  Soldiers  with  IMT  data  at  the  time  data  were  extracted.  NRestartej=  number  of  Soldiers  who  restarted  at  least  once  during  IMT.  %Restarted  =  percentage  of  Soldiers 
who  restarted  at  least  once  during  IMT  [(NRestarteci/N)  x  100],  Standardization  excludes  MOS  with  insufficient  sample  size. 
b  N=  number  of  Soldiers  with  AIT  school  grade  data  at  time  data  were  extracted. 


Criterion  Composites 


A  number  of  the  criterion  scales  measure  similar  underlying  constructs.  Composites  of  these 
criterion  scales  were  developed  to  reduce  the  number  of  criteria  used  to  validate  the  TAPAS  and 
simplify  the  interpretation  of  results,  without  sacrificing  infonnation.  Composites  were  constructed 
using  theoretical  rationale  and  examined  using  Confirmatory  Factor  Analysis  (CFA).  We  also 
considered  policy-based  reasons  in  deciding  whether  to  keep  a  variable  as  a  standalone  criterion  or 
to  include  it  in  a  composite.  Criteria  of  high  interest  to  ARI  and  the  Army  (e.g.,  those  of  most 
interest  to  stakeholders)  were  left  as  standalone  criteria.  Table  4.14  lists  the  IMT  and  in-unit 
criterion  composites,  the  scales  included  in  each  composite,  and  a  brief  description  of  how  the 
composite  was  constructed.  We  attempted  to  keep  the  in-unit  criteria  similar  to  the  IMT  criteria. 

The  results  of  the  confirmatory  factor  analyses  showed  strong  support  for  the  criterion 
composites  (see  Table  4.15).  All  of  the  fit-indices  were  within  the  range  of  acceptable  values. 
Further,  the  average  correlations  among  the  corresponding  scales  of  the  composites  were  high. 
Descriptive  statistics  for  the  IMT  and  in-unit  criterion  composites  are  shown  in  Table  4.16. 
Composite  reliability  estimates  were  computed  for  the  combined  Tier  1  and  Tier  2  sample  for 
composites  comprising  more  than  one  scale.  Results  were  generally  similar  to  the  stand  alone 
versions  of  the  scales.  Appendix  B  presents  correlations  among  the  criterion  composites. 


Table  4.14.  IMT  and  In-Unit  Criterion  Scores 


IMT 


Criterion  Score 

Scales 

Description 

Overall  Performance 

PRS:  Army  Adjustment 

PRS:  Effort  and  Discipline 

PRS:  MOS  Qualification 

PRS:  Physical  Fitness 

PRS:  Working  with  Others 

PRS:  MOS  Specific 

General  effort/  motivation  criterion.  Scales  are 
averaged  to  form  the  composite. 

Commitment  &  Fit 

ALQ:  Affective  Commitment 
ALQ:  Normative  Commitment 
ALQ:  General  MOS  Fit 

ALQ:  Needs  Supplies  Army  Fit 

General  commitment  to  and  fit  with  the  Army. 
Scales  are  averaged  to  form  the  composite. 

Retention  Cognitions 

ALQ:  Army  Career  Intentions 
ALQ:  Army  Re-enlistment 

ALQ:  Attrition  Cognition 

General  intentions  of  continuance  in  the  Army. 
Scales  are  averaged  to  form  the  composite. 

Army  Life  Adjustment 

ALQ:  Army  Life  Adjustment 

Army  life  adjustment  remains  a  standalone  criterion. 

Physical  Fitness 

ALQ:  Last  APFT  Score 

Physical  fitness  remains  a  standalone  criterion. 

Knowledge  &  Skill 

Army-Wide  JKT 

MOS  JKT 

AIT  Grade 

AW  JKT  and  MOS  JKT  are  averaged  to  form  an 
overall  knowledge/skill  composite.  For  those  that  do 
not  have  an  MOS  JKT  score,  AIT  grade  is  substituted. 

Disciplinary  Incidents 

ALQ:  Disciplinary  Incidents 

A  dichotomous  version  of  Disciplinary  Incidents 
was  constructed  based  on  whether  a  Soldier 
indicated  at  least  one  disciplinary  incident. 

Training  Restarts 

ALQ:  Training  Restarts 

ATRRS:  Training  Restarts 

If  either  the  ALQ  or  ATRRS  indicates  a  training 
restart,  then  Soldier  is  identified  as  having  a  restart. 
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Table  4.14.  (Continued) 


In-Unit 

Criterion  Score 

Scales 

Description 

Attrition 

6-months  -  30  months  attrition 

Attrition  status  remains  as  standalone  criteria. 

Overall  Performance 

PRS:  Can  Do 

PRS:  Effort  and  Personal  Discipline 
PRS:  Working  with  Others 

PRS:  Self-Management 

PRS:  Physical  Fitness  and  Bearing 

General  effort/  motivation  criterion.  Scales  are 
averaged  to  form  the  composite. 

Leadership  Potential 

PRS:  Overall  Leadership  Potential 

A  Soldier’s  potential  for  leadership  remains  a 
standalone  criterion. 

Commitment  &  Fit 

ALQ:  Affective  Commitment 

ALQ:  General  MOS  Fit 

ALQ:  Needs  Supplies  Army  Fit 

General  commitment  to  and  fit  with  the  Army. 

Scales  are  averaged  to  form  the  composite. 

Retention  Cognitions 

ALQ:  Army  Career  Intentions 

ALQ:  Army  Re-enlistment 

ALQ:  Attrition  Cognition 

General  intentions  of  continuance  in  the  Army. 

Scales  are  averaged  to  form  the  composite. 

Physical  Fitness 

ALQ:  Last  APFT  Score 

Physical  fitness  remains  a  standalone  criterion. 

Knowledge  &  Skill 

Army- Wide  JKT 

MOS  JKT 

AW  JKT  and  MOS  JKT  are  averaged  to  form  an 
overall  knowledge/skill  composite.  For  those  that 
do  not  have  MOS  JKT,  only  AW  JKT  scores  are 
used. 

Disciplinary  Incidents 

ALQ:  Disciplinary  Incidents 

A  dichotomous  version  of  Disciplinary  Incidents 
was  constructed  based  on  whether  a  Soldier 
indicated  at  least  one  disciplinary  incident. 

Table  4.15.  Criterion  Composite  Confirmatory  Factor  Analysis  (CFA)  Model  Results 


n 

2 

X 

RMSEA 

CFI 

TLI 

SRMR 

avg.  r 

IMT 

Commitment/  Fit 

41,611 

35.56 

0.02 

1.00 

1.00 

0.00 

0.60 

Overall  Performance 

15,353 

680.60 

0.07 

0.99 

0.98 

0.02 

0.67 

Retention  Cog 

41,611 

0.00 

0.00 

1.00 

1.00 

0.00 

0.61 

Knowledge/Skill 

40,128 

— 

— 

— 

— 

— 

0.54 

In-Unit 

Commitment  &  Fit 

2,509 

0.00 

0.00 

1.00 

1.00 

0.00 

0.52 

Overall  Performance 

1,371 

12.17 

0.03 

1.00 

1.00 

0.01 

0.69 

Retention  Cog 

2,509 

0.00 

0.00 

1.00 

1.00 

0.00 

0.57 

Knowledge/Skill 

2,486 

— 

- 

— 

- 

— 

0.45 

Note.  RMSEA  =  Root  mean  square  error  of  approximation,  acceptable  values  are  <  .08;  SRMR  =  Standardized  root  mean  square 
residual,  acceptable  values  are  <  .08;  CFI  =  Comparative  Fit  Index,  acceptable  values  are  >  .95;  TLI  =  Tucker-Lewis  Index, 
acceptable  values  are  >  .95  (Hu  &  Bentler,  1999).  CFA  models  were  only  fit  to  criterion  with  greater  than  three  scales. 
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Table  4.16.  Descriptive  Statistics  for  Criterion  Composites  by  Education  Tier  in  the  IMT  and 


In-Unit  Validation  Samples 


Domain/Setting/MOS 

n 

M 

SD 

Min 

Max 

a 

Tier  1 

IMT  Composites 

Overall  Performance 

4,400 

3.31 

.78 

1.00 

5.00 

— 

Physical  Fitness 

14,579 

252.26 

30.65 

109.00 

300.00 

~ 

Commitment  &  Fit 

14,723 

3.98 

.58 

1.13 

5.00 

— 

Retention  Cognitions 

14,723 

2.71 

.58 

1.00 

4.33 

- 

Knowledge  &  Skill  a 

11,895 

.05 

.85 

-3.83 

2.27 

~ 

Training  Restarts  (Y/N) 

69,038 

.18 

.39 

.00 

1.00 

- 

Disciplinary  Incidents  (Y/N) 

13,168 

.21 

.41 

.00 

1.00 

- 

Army  Life  Adjustment 

16,201 

4.09 

.66 

1.00 

5.00 

- 

In-Unit  Composites 

Overall  Performance 

664 

5.24 

1.12 

2.02 

7.00 

— 

Physical  Fitness 

923 

246.61 

31.78 

120.00 

300.00 

— 

Commitment  &  Fit 

944 

3.62 

.66 

1.51 

5.00 

— 

Retention  Cognitions 

943 

2.47 

.64 

1.00 

3.83 

- 

Knowledge  &  Skill  a 

936 

.06 

.90 

-3.64 

2.29 

- 

Leadership  Potential 

650 

4.88 

1.67 

1.00 

7.00 

- 

Disciplinary  Incidents  (Y/N) 

944 

.21 

.41 

.00 

1.00 

- 

Tier  2 

b 

IMT  Composites 

Overall  Performance 

151 

3.40 

.70 

1.58 

4.84 

— 

Physical  Fitness 

566 

245.09 

32.42 

124.00 

300.00 

— 

Commitment  &  Fit 

571 

4.05 

.59 

1.69 

5.00 

— 

Retention  Cognitions 

571 

2.80 

.59 

1.00 

3.83 

- 

Knowledge  &  Skill11 

458 

.05 

.87 

-3.09 

2.14 

- 

Training  Restarts  (Y/N) 

2,622 

.22 

.41 

.00 

1.00 

- 

Disciplinary  Incidents  (Y/N) 

538 

.23 

.42 

.00 

1.00 

- 

Army  Life  Adjustment 

634 

4.12 

.67 

1.22 

5.00 

- 

Tier  1  +  Tier  2  (Combined) 

IMT  Composites 

Overall  Performance 

4,551 

3.31 

.78 

1.00 

5.00 

- 

Physical  Fitness 

15,145 

252.00 

30.75 

109.00 

300.00 

- 

Commitment  &  Fit 

15,294 

3.99 

.58 

1.13 

5.00 

.95 

Retention  Cognitions 

15,294 

2.72 

.58 

1.00 

4.33 

.89 

Knowledge  &  Skill  a 

12,353 

.05 

.85 

-3.83 

2.27 

.82 

Training  Restarts  (Y/N) 

71,660 

.18 

.39 

.00 

1.00 

- 

Disciplinary  Incidents  (Y/N) 

13,706 

.21 

.41 

.00 

1.00 

- 

Army  Life  Adjustment 

16,835 

4.09 

.66 

1.00 

5.00 

.87 

In-Unit  Composites 

Overall  PRS 

674 

5.22 

1.13 

2.02 

7.00 

- 

Physical  Fitness 

939 

246.53 

31.71 

120.00 

300.00 

- 

Commitment  &  Fit 

960 

3.62 

.66 

1.51 

5.00 

.94 

Retention  Cognitions 

959 

2.47 

.65 

1.00 

3.83 

.86 

Knowledge  &  Skill  a 

951 

.06 

.90 

-3.64 

2.29 

.75 

Leadership  Potential 

660 

4.87 

1.66 

1.00 

7.00 

- 

Disciplinary  Incidents  (Y/N) 

960 

.22 

.41 

.00 

1.00 

- 

a  Knowledge  &  Skill  is  reported  on  a  standardized  z-score  metric. 

b  Sample  size  for  Tier  2  in-unit  composites  were  all  less  than  50  so  no  results  are  reported. 
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Summary 


Criterion  data,  such  as  attrition,  training  restarts,  and  AIT  course  grades,  were  gathered  from 
administrative  records.  In  addition,  three  types  of  criterion  measures  were  adapted  from  previous 
Anny  research  to  validate  the  TAPAS  and  ICTL:  (a)  the  JKTs,  (b)  the  PRS,  and  (c)  the  ALQ. 

The  JKTs  measure  WTBD  (Army-wide)  and  (for  eight  target  MOS)  MOS-specific  knowledge. 
These  were  combined  with  administrative  records  of  AIT  grades  to  fonn  a  Knowledge/Skill 
composite,  intended  to  measure  a  Soldier’s  task-specific  knowledge.  The  PRS  are  completed  by 
training  cadre  (IMT)  or  supervisors  (in-unit)  and  measure  Army-wide  constructs  such  as  effort 
and  leadership  and  (for  selected  IMT  MOS)  MOS-specific  competence.  The  PRS  were  combined 
to  fonn  an  Overall  Performance  composite  intended  to  measure  cadre  and/or  supervisor  ratings 
of  a  Soldier’s  general  performance  level.  Third,  the  ALQ  asks  Soldiers  to  complete  verifiable 
self-report  perfonnance  items  (e.g.,  their  APFT  scores)  and  self-report  attitudinal  items  (e.g., 
adjustment  to  Army  life).  For  the  validation  analyses,  the  ALQ  scales  were  combined  to  form  a 
Commitment  and  Fit  composite  and  a  Retention  Cognitions  composite.  Several  other  scales  were 
left  as  standalone  criteria. 

In  general,  the  criterion  measures  described  in  this  chapter  exhibited  acceptable  and  theoretically 
consistent  psychometric  properties.  The  exception  to  this  was  the  Army-wide  and  MOS-specific 
PRS,  which  continued  to  exhibit  low  interrater  reliability  coefficients.  Results  involving  the  PRS 
should  be  interpreted  with  caution. 


31 


CHAPTER  5:  EVIDENCE  FOR  THE  PREDICTIVE  VALIDITY  OF  THE  TAPAS 


Joseph  Caramagno  (HumRRO) 


This  chapter  evaluates  the  potential  of  the  TAPAS  to  predict  Soldiers’  performance  and  retention 
through  their  first  unit  of  assignment.  We  begin  with  a  brief  description  of  our  analytic  approach. 
Next,  we  summarize  the  main  findings  from  (a)  incremental  validity  analyses  involving  the 
existing  operational  and  newly  developed  TOPS  composites  (Can-Do,  Will-Do,  Adaptation)  and 
a  set  of  optimally  weighted  scores  on  nine  TAPAS  dimensions  that  are  common  across  the 
TAPAS  versions;  and  (b)  bivariate  correlations  between  the  TAPAS  scales  and  selected 
performance  and  retention  criteria. 


Analysis  Approach 

To  evaluate  the  TAPAS’  potential  to  enhance  new  Soldier  selection,  we  examined  the 
incremental  validity  of  the  TAPAS  over  the  AFQT  in  predicting  first-term  outcomes  important  to 
the  Anny.  Consistent  with  the  Anny’s  personnel  goals,  we  examined  performance  and  retention- 
related  outcomes  that,  as  a  group,  provide  representative  coverage  of  the  criterion  space 
(Campbell,  Hanson,  &  Oppler,  2001;  Campbell,  McHenry,  &  Wise,  1990;  Knapp  &  Tremble, 
2007;  Strickland,  2005).  In  this  sixth  analysis  cycle,  we  included  newly  developed  criterion 
composites  as  well  as  a  select  number  of  stand-alone  criteria  to  reduce  the  criterion  space  while 
also  accounting  for  overlap  among  the  outcome  measures. 

Our  analysis  approach  was  generally  consistent  with  previous  evaluations  of  the  TAPAS  and 
similar  experimental  non-cognitive  predictors  (e.g.,  Ingerick  et  ah,  2009;  Knapp  &  Heffner, 

2009;  2010;  Trippe,  Caramagno,  Allen,  &  Ingerick,  2011).  In  brief,  this  approach  involved 
testing  a  series  of  two-step  hierarchical  regression  models,  where  scores  on  each  criterion 
measure  or  composite  were  regressed  onto  Soldiers’  AFQT  scores  in  the  first  step,  followed  by 
scores  on  the  TOPS  composites  or  TAPAS  scales  in  the  second  step.  The  resulting  increment  in 
the  multiple  correlation  (A R)  from  adding  the  TAPAS  in  the  second  step  served  as  the  index  of 
incremental  validity. 

For  the  continuously  scaled  criteria,  the  models  were  estimated  using  Ordinary  Least  Squares 
(OLS)  regression.  Logistic  regression  was  used  for  the  dichotomous  criteria  (i.e.,  attrition, 
disciplinary  incidents,  training  restart).  For  the  logistic  regression  models,  we  estimated  point- 
biserial  correlations  (rpk)  in  place  of  the  traditional  pseudo-/?  estimates.  These  point -biserial 
correlations  reflected  the  correlation  between  a  Soldier’s  predicted  probability  of  exhibiting  a 
selected  behavior  and  his  or  her  actual  behavior  (e.g.,  attriting).  We  estimated  these  correlations 
because  of  the  well-known  conceptual  and  statistical  issues  associated  with  traditional  pseudo- 
R  estimates  (Cohen,  Cohen,  West,  &  Aiken,  2003). 

A  series  of  six  regression  models  were  estimated  for  each  criterion  measure  (where  k  =  the 
number  of  predictors  in  the  model):  (a)  a  model  consisting  of  TAPAS  scales  common  across  the 
TOPS  composites  (k  =  9),  (b)  the  original  (current  operational)  TOPS  Can-Do  composite  (k  =  1), 
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(c)  the  operational  TOPS  Will-Do  composite  ( k  =  1),  (d)  the  revised  TOPS  Can-Do  composite  (/ k 
=  1),  (e)  the  revised  TOPS  Will-Do  composite  ( k  =  1),  and  (f)  the  new  TOPS  Adaptation 
composite  (k  =  1).  Estimates  for  the  first  model,  the  model  consisting  of  all  TAPAS  scales, 
represent  the  best-case  scenario  of  the  TAPAS  predictive  potential,  whereas  estimates  for  the 
TOPS  composite  models  capture  the  predictive  potential  of  the  composites  as  configured  for 
operational  usage.  Table  5.1  provides  a  summary  of  each  of  the  regression  models.  All  models 
were  estimated  separately  by  education  tier,  in  addition  to  Tiers  1  and  2  combined  where  sample 
size  allowed. 


Table  5.1.  Summary  of  the  Regression  Models 


Model  Name 

#  of  Predictors  (£) 

Description 

Common  TAPAS  Scales 

9 

Model  consists  of  a  set  of  optimally  weighted  scores  on 

9  TAPAS  dimensions  (or  facets)  that  are  common 
across  all  TOPS  composites. 

Original  TOPS  Can-Do  Composite 

1 

Model  consists  of  scores  on  the  existing  TOPS  Can-Do 
composite. 

Original  TOPS  Will-Do  Composite 

1 

Model  consists  of  scores  on  the  existing  TOPS  Will-Do 
composite. 

Revised  TOPS  Can-Do  Composite 

1 

Model  consists  of  scores  on  the  revised  TOPS  Can-Do 
composite. 

Revised  TOPS  Will-Do  Composite 

1 

Model  consists  of  scores  on  the  revised  TOPS  Will-Do 
composite. 

New  TOPS  Adaptation  Composite 

1 

Model  consists  of  scores  on  the  newly  created  TOPS 
Adaptation  composite. 

To  enable  comparisons  across  the  different  models  and  education  tier  levels,  we  adjusted  the 
observed  R  and  A R  for  shrinkage.  Comparing  the  aforementioned  models  directly,  without  the 
shrinkage  adjustments,  could  have  led  to  incorrect  conclusions.  The  reason  for  this  is  because  the 
observed  R  becomes  inflated  as  more  predictors  are  entered  in  the  model,  regardless  of  the  “true” 
relationship  between  the  predictors  and  the  criterion.  This  over  inflation  can  be  particularly 
problematic  when  the  sample  size  is  small,  as  is  the  case  with  several  of  the  Tier  2  predictor- 
criterion  relationships.  Adjusting  the  estimates  for  shrinkage  enables  a  direct  comparison  across 
models  for  the  same  criterion  or  by  education  tier.  We  adjusted  the  observed  R  and  A R  estimates 
using  Burket’s  (1964)  formula  for  shrinkage  (cf.  Formula  8;  Schmitt  &  Ployhart,  1999).  This 
adjustment  was  implemented  as  follows: 

1 .  Using  the  observed  (unadjusted)  correlations  among  the  TAPAS,  AFQT,  or  education 
tier  and  the  selected  criterion  previously  estimated,  correlations  between  the 
predictors  and  the  performance-related  criteria  (e.g.,  JKT)  were  adjusted  for  sample 
size  and  number  of  predictors  using  Burket’s  (1964)  fonnula  for  shrinkage: 

pc  =  (NR2  -k)l[R(N-k)}  (1) 

where  k  equals  the  number  of  elements  in  the  model. 
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2.  The  shrinkage-adjusted  incremental  validity  estimates  for  the  experimental  predictors 
were  calculated  by  subtracting  the  adjusted  R  associated  with  the  AFQT-only  model 
from  the  adjusted  R  obtained  from  the  full  model  (i.e.,  the  AFQT  +  Experimental 
Predictor  model). 

In  addition  to  the  incremental  validity  analyses,  we  examined  the  predictive  validity  of  the 
individual  TAPAS  scales  based  on  the  bivariate  correlations  between  scores  on  the  TAPAS 
scales  and  the  selected  criterion  measures.  The  results  of  these  analyses  are  presented  in 
Appendix  D. 


Findings 

Findings  using  the  new  criterion  composites  and  select  single  measure  predictors  are  organized 
by  criterion  domain:  (a)  IMT  perfonnance,  (b)  in-unit  perfonnance,  and  (c)  attrition  (data 
captured  quarterly).  A  few  notes  related  to  interpretation  of  the  findings: 

•  The  interrater  reliability  estimates  for  the  PRS  were  generally  low.  Therefore,  the 
predictive  validity  evidence  associated  with  the  PRS  (e.g.,  Overall  Perfonnance)  should 
be  interpreted  with  caution. 

•  The  results  for  Tier  2  Soldiers  should  be  interpreted  with  caution  at  this  stage  of  the 
TOPS  evaluation  because  of  limited  criterion  data  for  those  Soldiers.  Accordingly,  our 
discussion  primarily  focuses  on  the  shrinkage  adjusted  results  for  Tier  1  and  2  Soldiers 
(combined).9 

•  The  shrinkage  adjusted  results  presented  in  this  report  should  not  be  directly  compared  to 
previously  reported  results  that  were  not  adjusted  for  shrinkage. 

•  Most  of  our  discussion  focuses  on  the  models  involving  the  TOPS  composites  because 
these  models  best  evaluate  the  TAPAS’  current  operational  format  as  well  as  its  potential 
future  fonnat.  However,  it  should  be  noted  that  models  based  on  an  optimally  weighted 
composite  of  nine  common  TAPAS  dimensions  yielded  incremental  validity  estimates 
that  were  comparable  to  or  higher  than  the  reduced  models,  on  average,  after  adjusting 
the  estimates  for  shrinkage. 


9  Results  for  Tier  1  and  Tier  2  applicants  combined  were  generally  comparable  to  Tier  1  only  results. 
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Predicting  IMT  Performance 


Tables  5.2  to  5.4  summarize  the  incremental  validity  results  for  predicting  IMT  performance 
criteria  over  and  above  the  AFQT.  Consistent  with  expectations  and  previous  analyses,  the  TOPS 
composites  (Can-Do  and  Will-Do;  original  and  revised)  and  common  TAPAS  scales  evidenced 
no  notable  increments  over  the  AFQT  in  predicting  scores  on  the  composite  measure  of  technical 
knowledge  and  skill  (Adj.  A R  were  consistently  less  than  .01).  In  contrast,  the  common  TAPAS 
scales  and  the  TOPS  composites  exhibited  small  to  moderate  gains  in  predictive  validity  over  the 
AFQT  in  predicting  outcomes  on  several  important  nontechnical  perfonnance  criteria.  For 
example,  the  common  TAPAS  scales  and  the  revised  Will-Do  composite  demonstrated  modest 
potential  to  enhance  the  prediction  of  Physical  Fitness  (Adj.  A R  =  .21  and  .18,  respectively), 
Army  Life  Adjustment  (Adj.  A R  =  .15  and  .14,  respectively),  Commitment  and  Fit  (Adj.  A R  = 

.13  and  .12,  respectively),  and  Disciplinary  Incidents  (Adj.  AR  =  .12  and  .11,  respectively).  The 
original  Can-Do  composite  also  exhibited  small  gains  over  the  AFQT  in  predicting  Soldiers’ 
attitudes  towards  the  Army  (Commitment  and  Fit,  Adj.  A R  =  .09;  Army  Life  Adjustment,  Adj. 

AR  =  .08). 


None  of  the  original  or  revised  TOPS  composites  contributed  substantially  to  the  prediction  of 
cadre  ratings  of  overall  perfonnance  (Adj.  AR  ranged  from  .00  to  .06),  Soldiers’  retention 
cognitions  (Adj.  AR  ranged  from  .00  to  .02),  or  the  incidence  of  recycling  at  least  once  during 
training  (Adj.  AR  ranged  from  .00  to  .04).  The  largest  increment  over  the  AFQT  in  predicting 
these  criteria  was  observed  for  the  model  containing  the  common  TAPAS  dimensions  (Adj.  AR 
ranged  from  .04  to  .06). 
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Table  5.2.  Incremental  Validity  Estimates  for  the  TAPAS  over  AFQT  for  Predicting  IMT  Technical  Performance  and  Discipline-related 
Criteria  by  Education  Tier 


Tier  1  Tier  2  Tier  1+2  (Combined) 


IMT  Criterion  Measure  /  Model 

AFQT 

R 

AFQT  + 
TAPAS  R 

A  R 

Adj 

AR 

AFQT 

R 

AFQT  + 
TAPAS  R 

AR 

Adj 

AR 

AFQT 

R 

AFQT  + 
TAPAS  R 

AR 

Adj 

AR 

Knowledge  &  Skill 

n  =  10,267 

-11,895 

n  =  340  - 

458 

n  =  10607- 

- 12,353 

Common  TAPAS  Scales  (9) 

.46 

.46 

.00 

.00 

.30 

.32 

.02 

.00 

.46 

.46 

.00 

.00 

Original  Can-Do 

.46 

.46 

.00 

.00 

.30 

.30 

.00 

.00 

.46 

.46 

.00 

.00 

Original  Will-Do 

.46 

.46 

.00 

.00 

.30 

.30 

.00 

.00 

.46 

.46 

.00 

.00 

Revised  Can-Do 

.46 

.46 

.00 

.00 

.30 

.30 

.00 

.00 

.46 

.46 

.00 

.00 

Revised  Will-Do 

.46 

.46 

.00 

.00 

.30 

.31 

.00 

.00 

.46 

.46 

.00 

.00 

New  Adaptation3 

.46 

.46 

.00 

.00 

.30 

.31 

.01 

.00 

.46 

.46 

.00 

.00 

Disciplinary’  Incidents 

n  =  7,228  ■ 

-8,167 

n  =  209  - 

281 

n  =  7,437- 

-  8,448 

Common  TAPAS  Scales  (9) 

.02 

.15 

.12 

.12 

.04 

.17 

.14 

.00 

.02 

.14 

.12 

.12 

Original  Can-Do 

.03 

.08 

.05 

.06 

.09 

.09 

.00 

.00 

.03 

.08 

.05 

.05 

Original  Will-Do 

.02 

.11 

.08 

.09 

.04 

.09 

.05 

.01 

.02 

.11 

.08 

.08 

Revised  Can-Do 

.03 

.04 

.01 

.01 

.09 

.11 

.02 

.00 

.03 

.04 

.01 

.01 

Revised  Will-Do 

.02 

.14 

.11 

.12 

.04 

.12 

.08 

.05 

.02 

.14 

.11 

.11 

New  Adaptation  a 

.03 

.10 

.07 

.07 

.09 

.09 

.00 

.00 

.03 

.10 

.07 

.07 

Training  Restart 

n  =37,676- 

-  39,269 

n  =  1,004- 

1,101 

n  =  38,680  - 

-40,370 

Common  TAPAS  Scales  (9) 

.02 

.06 

.04 

.04 

.01 

.09 

.08 

.00 

.02 

.06 

.04 

.04 

Original  Can-Do 

.02 

.02 

.00 

.00 

.01 

.01 

.00 

.00 

.02 

.02 

.00 

.00 

Original  Will-Do 

.02 

.03 

.01 

.01 

.01 

.02 

.01 

.00 

.02 

.03 

.01 

.01 

Revised  Can-Do 

.02 

.02 

.00 

.00 

.01 

.04 

.03 

.00 

.02 

.02 

.00 

.00 

Revised  Will-Do 

.02 

.05 

.03 

.03 

.01 

.01 

.00 

.00 

.02 

.05 

.03 

.03 

New  Adaptation  a 

.02 

.04 

.02 

.03 

.01 

.01 

.00 

.00 

.02 

.04 

.02 

.02 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT..  AFQT  +  TAPAS  =  Multiple 
correlation  ( R )  between  the  AFQT  +  selected  TAPAS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected 
TAPAS/TOPS  composite  scales  over  AFQT  to  the  regression  model  ([AFQT  +  TAPAS]  -  AFQT  Only).  Estimates  in  bold  were  statistically  significant,/?  <  .05 
(one-tailed).  Note  that  significance  tests  cannot  be  applied  to  the  adjusted  AR. 


Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Table  5.3.  Incremental  Validity  Estimates  for  the  TAPAS  over  AFQT  for  Predicting  IMT  Adjustment,  Commitment  and  Fit,  and 
Retention  Criteria  by  Education  Tier 


Tier  1  Tier  2  Tier  1  +  2  (Combined) 


IMT  Criterion  Measure  /  Model 

AFQT 

R 

AFQT  + 
TAPAS  R 

A R 

Adj 

AR 

AFQT 

R 

AFQT  + 
TAPAS  R 

AR 

Adj 

AR 

AFQT 

R 

AFQT  + 
TAPAS  R 

AR 

Adj 

AR 

Army  Life  Adjustment 

n  =  12,717- 

14,723 

n  =  424  - 

571 

n  =  13,141  - 

- 15,294 

Common  TAPAS  Scales  (9) 

.07 

.22 

.15 

.15 

.01 

.19 

.18 

.11 

.07 

.22 

.15 

.15 

Original  Can-Do 

.07 

.15 

.08 

.08 

.01 

.12 

.11 

.08 

.07 

.15 

.08 

.08 

Original  Will-Do 

.07 

.12 

.05 

.05 

.01 

.09 

.08 

.05 

.07 

.12 

.05 

.05 

Revised  Can-Do 

.07 

.10 

.03 

.03 

.01 

.03 

.01 

.00 

.07 

.09 

.03 

.03 

Revised  Will-Do 

.07 

.20 

.14 

.14 

.01 

.18 

.17 

.16 

.07 

.20 

.14 

.14 

New  Adaptation  a 

.07 

.11 

.04 

.04 

.01 

.07 

.05 

.00 

.07 

.11 

.05 

.04 

Commitment  &  Fit 

n  =  12,717- 

14,723 

n  =  424  - 

571 

n  =13,141- 

-  15,294 

Common  TAPAS  Scales  (9) 

.01 

.14 

.13 

.13 

.02 

.17 

.15 

.07 

.01 

.14 

.13 

.13 

Original  Can-Do 

.01 

.09 

.08 

.09 

.02 

.09 

.07 

.04 

.01 

.09 

.08 

.09 

Original  Will-Do 

.01 

.08 

.07 

.08 

.02 

.06 

.04 

.01 

.01 

.08 

.07 

.08 

Revised  Can-Do 

.01 

.02 

.02 

.02 

.02 

.05 

.02 

.00 

.01 

.02 

.02 

.02 

Revised  Will-Do 

.01 

.11 

.11 

.12 

.02 

.14 

.12 

.11 

.01 

.11 

.11 

.12 

New  Adaptation  a 

.01 

.04 

.03 

.04 

.02 

.03 

.01 

.00 

.01 

.04 

.03 

.04 

Retention  Cognitions 

n  =  12,717- 

14,723 

n  =  424  - 

571 

n  =  13,141  - 

- 15,294 

Common  TAPAS  Scales  (9) 

.11 

.16 

.04 

.04 

.14 

.20 

.05 

.00 

.11 

.16 

.04 

.04 

Original  Can-Do 

.11 

.14 

.02 

.02 

.14 

.15 

.00 

.00 

.11 

.14 

.02 

.02 

Original  Will-Do 

.11 

.13 

.01 

.01 

.14 

.14 

.00 

.00 

.11 

.13 

.01 

.01 

Revised  Can-Do 

.11 

.12 

.01 

.01 

.14 

.14 

.00 

.00 

.11 

.12 

.01 

.01 

Revised  Will-Do 

.11 

.12 

.00 

.00 

.14 

.16 

.02 

.01 

.11 

.12 

.00 

.00 

New  Adaptation  a 

.11 

.12 

.00 

.00 

.14 

.15 

.00 

.00 

.11 

.12 

.00 

.00 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT..  AFQT  +  TAPAS  =  Multiple 
correlation  (R)  between  the  AFQT  +  selected  TAPAS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected 
TAPAS/TOPS  composite  scales  over  AFQT  to  the  regression  model  ([AFQT  +  TAPAS]  -  AFQT  Only).  Estimates  in  bold  were  statistically  significant,/?  <  .05 
(one-tailed).  Note  that  significance  tests  cannot  be  applied  to  the  adjusted  AR. 


Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Table  5.4.  Incremental  Validity  Estimates  for  the  TAPAS  overAFQT  for  Predicting  IMT  Physical  Fitness  and  Overall  Performance 
Criteria  by  Education  Tier 


IMT  Criterion  Measure  /  Model 

Tier  1 

Tier  2 

Tier  1  +  2  (Combined) 

AFQT 

R 

AFQT  + 
TAPAS  R 

A R 

Adj 

AR 

AFQT 

R 

AFQT  + 
TAPAS  R 

AR 

Adj 

AR 

AFQT 

R 

AFQT  + 
TAPAS  R 

AR 

Adj 

AR 

Physical  Fitness 

n  =  12,588 

- 14,579 

n  =  420  - 

566 

n  =  13,008 

-15,145 

Common  TAPAS  Scales  (9) 

.09 

.30 

.21 

.21 

.06 

.29 

.23 

.21 

.09 

.30 

.22 

.21 

Original  Can-Do 

.09 

.09 

.00 

.00 

.06 

.10 

.04 

.03 

.09 

.09 

.00 

.00 

Original  Will-Do 

.09 

.12 

.03 

.03 

.06 

.12 

.06 

.06 

.09 

.12 

.03 

.03 

Revised  Can-Do 

.09 

.10 

.01 

.01 

.06 

.06 

.00 

.00 

.09 

.10 

.01 

.01 

Revised  Will-Do 

.09 

.27 

.18 

.18 

.06 

.28 

.22 

.24 

.09 

.27 

.18 

.18 

New  Adaptation  a 

.09 

.18 

.09 

.09 

.06 

.15 

.09 

.10 

.09 

.18 

.09 

.09 

Overall  Performance 

n  =  3,892 

-4,400 

n  =  105  - 

151 

n  =  3,997 

-4,551 

Common  TAPAS  Scales  (9) 

.08 

.15 

.07 

.06 

.03 

.23 

.21 

.00 

.08 

.15 

.07 

.06 

Original  Can-Do 

.08 

.08 

.00 

.00 

.03 

.03 

.00 

.00 

.08 

.08 

.00 

.00 

Original  Will-Do 

.08 

.10 

.02 

.02 

.03 

.04 

.01 

.00 

.08 

.10 

.02 

.02 

Revised  Can-Do 

.08 

.08 

.00 

.00 

.03 

.03 

.00 

.00 

.08 

.08 

.00 

.00 

Revised  Will-Do 

.08 

.13 

.05 

.05 

.03 

.15 

.12 

.06 

.08 

.13 

.05 

.05 

New  Adaptation  a 

.08 

.10 

.02 

.02 

.03 

.04 

.01 

.00 

.08 

.10 

.02 

.02 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT.  AFQT  +  TAPAS  =  Multiple 
correlation  ( R )  between  the  AFQT  +  selected  TAPAS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected 
TAPAS/TOPS  composite  scales  over  AFQT  to  the  regression  model  ([AFQT  +  TAPAS]  -  AFQT  Only).  Estimates  in  bold  were  statistically  significant,/?  <  .05 
(one-tailed).  Note  that  significance  tests  cannot  be  applied  to  the  adjusted  AR. 


Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Predicting  In-Unit  Performance 


The  incremental  validity  results  for  predicting  in-unit  perfonnance  are  presented  in  Tables  5.5  - 
5.7.  Separate  analyses  were  not  conducted  for  Tier  2  Soldiers  because  those  Soldiers  had  limited 
in-unit  criterion  data  ( n  <  50).  As  before,  our  discussion  focuses  on  the  shrinkage  adjusted  results 
for  Tier  1  and  2  Soldiers  combined.  None  of  the  predictor  measures  demonstrated  potential  to 
increment  the  AFQT  in  the  prediction  of  in-unit  technical  perfonnance  (incremental  validity 
estimates  associated  with  scores  on  the  Knowledge  and  Skill  measure  were  at  or  near  zero  for  all 
models).  The  TAPAS  scales  and  TOPS  composites  provided  limited  additional  infonnation 
beyond  the  AFQT  in  the  prediction  of  three  nontechnical  criteria  as  well  (Overall  Perfonnance 
Adj.  A R  <  .04,  Leadership  Potential  Adj.  A R  <  .04,  Retention  Cognitions  Adj.  A R  <  .03). 

The  largest  gains  in  predictive  validity  were  observed  for  models  involving  Physical  Fitness 
(self-reported  APFT  scores).  The  inclusion  of  the  common  TAPAS  scales  (Adj.  A R  =  .25), 
revised  Will-Do  composite  (Adj.  AR  =  .21),  and  Adaptation  composite  (Adj.  Ai?  =  .13)  in  the 
model  contributed  a  modest  amount  of  additional,  non-trivial  infonnation  (beyond  that  of  the 
AFQT  only).  In  addition,  the  adjusted  estimate  of  A R  for  the  revised  Will-Do  composite  was 
three  times  that  of  the  original  Will-Do  composite  (Adj.  A R  =  .07)  for  this  criterion.  The  TAPAS 
scales  and  TOPS  composites  significantly  incremented  the  AFQT  in  the  prediction  of 
disciplinary  incidents  as  well,  but  the  change  ini?  was  smaller  (original  Will-Do  composite  Adj. 
Ai?  =  09;  revised  Will-Do  composite  Adj.  Ai?  =  .07;  common  TAPAS  scales  Adj.  Ai?  =  .07; 
original  Can-Do  composite  Adj.  A R  =  .06).  Finally,  it  should  be  noted  that  the  original  Can-Do 
composite  demonstrated  the  greatest  potential  (compared  to  other  TOPS  composites)  to 
increment  the  AFQT  in  the  prediction  of  Commitment  and  Fit  (Adj.  A R  =  .08);  however,  the 
magnitude  of  the  estimate  is  relatively  small. 

Predicting  Attrition 

Results  of  the  evaluation  of  the  TAPAS’  potential  to  increment  the  prediction  of  first-term 
Soldier  attrition  over  and  above  the  AFQT  are  displayed  in  Table  5.8.  Results  are  displayed  for 
6-,  12-,  24-,  and  30-month  attrition.  Although  all  of  the  estimates  are  statistically  significant, 
none  of  the  predictor  measures  exhibited  remarkable  gains  in  incremental  validity  beyond  the 
AFQT  (Adj.  A/?  ranged  from  .00  to  .04).  This  was  true  for  the  adjusted  and  unadjusted  estimates 
of  the  A R. 
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Table  5.5.  Incremental  Validity  Estimates  for  the  TAPAS  over  AFQT  for  Predicting  In-Unit  Technical  Performance  and 
Disciplineby  Education  Tier 


Tier  1 

Tier  1  +  2  (Combined) 

In-Unit  Criterion  Measure  /  Model 

AFQT  R 

AFQT  + 
TAPAS  R 

A  R 

Adj  AR 

AFQT  R 

AFQT  + 
TAPAS  R 

AR 

Adj  AR 

Knowledge  &  Skill 

n  =  921  - 

936 

n  =  935  - 

951 

Common  TAPAS  Scales  (9) 

.47 

.49 

.02 

.01 

.48 

.50 

.02 

.01 

Original  Can-Do 

.47 

.48 

.01 

.01 

.48 

.48 

.01 

.01 

Original  Will-Do 

.47 

.48 

.00 

.00 

.48 

.48 

.00 

.00 

Revised  Can-Do 

.47 

.48 

.01 

.01 

.48 

.49 

.01 

.01 

Revised  Will-Do 

.47 

.47 

.00 

.00 

.48 

.48 

.00 

.00 

New  Adaptation  a 

.47 

.47 

.00 

.00 

.48 

.48 

.00 

.00 

Disciplinary’  Incidents 

n  =  925  - 

940 

n  =  940  - 

956 

Common  TAPAS  Scales  (9) 

.02 

.13 

.11 

.06 

.01 

.13 

.13 

.07 

Original  Can-Do 

.01 

.08 

.07 

.06 

.00 

.08 

.08 

.06 

Original  Will-Do 

.02 

.11 

.09 

.09 

.01 

.11 

.10 

.09 

Revised  Can-Do 

.01 

.03 

.02 

.00 

.00 

.04 

.03 

.00 

Revised  Will-Do 

.02 

.08 

.06 

.05 

.01 

.09 

.08 

.07 

New  Adaptation  a 

.01 

.04 

.02 

.00 

.00 

.04 

.04 

.00 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT.  AFQT  +  TAPAS  =  Multiple 
correlation  ( R )  between  the  AFQT  +  selected  TAPAS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected 
TAPAS/TOPS  composite  scales  over  AFQT  to  the  regression  model  ([AFQT  +  TAPAS]  -  AFQT  Only).  Estimates  in  bold  were  statistically  significant,/?  <  .05 
(one-tailed).  Note  that  significance  tests  cannot  be  applied  to  the  adjusted  AR. 


Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Table  5.6.  Incremental  Validity  Estimates  for  the  TAPAS  over  AFQT  for  Predicting  In-Unit  Overall  Leadership  Potential, 
Commitment  and  Fit,  and  Retention  Criteria  by  Education  Tier 


In-Unit  Criterion  Measure  /  Model 

Tier  1 

Tier  1+2  (Combined) 

AFQT  R 

AFQT  + 
TAPAS  R 

A R 

Adj  A R 

AFQT  R 

AFQT  +  A„ 

TAPAS  R 

Adj  A R 

Leadership  Potential 

n  =  640  - 

650 

n  =  650  -  660 

Common  TAPAS  Scales  (9) 

.07 

.16 

.09 

.02 

.07 

.16  .09 

.02 

Original  Can-Do 

.07 

.07 

.00 

.00 

.07 

.07  .00 

.00 

Original  Will-Do 

.07 

.11 

.04 

.03 

.07 

.11  .04 

.04 

Revised  Can-Do 

.07 

.08 

.01 

.00 

.07 

.08  .01 

.00 

Revised  Will-Do 

.07 

.11 

.04 

.03 

.07 

.11  .04 

.03 

New  Adaptation  a 

.07 

.10 

.03 

.02 

.07 

.09  .03 

.02 

Commitment  &  Fit 

n  =  929  - 

944 

n  =  944  -  960 

Common  TAPAS  Scales  (9) 

.03 

.11 

.08 

.02 

.04 

.12  .08 

.02 

Original  Can-Do 

.03 

.11 

.08 

.09 

.04 

.11  .07 

.08 

Original  Will-Do 

.03 

.08 

.04 

.05 

.04 

.08  .04 

.04 

Revised  Can-Do 

.03 

.05 

.02 

.01 

.04 

.06  .02 

.01 

Revised  Will-Do 

.03 

.08 

.04 

.05 

.04 

.08  .04 

.05 

New  Adaptation  a 

.03 

.04 

.00 

.00 

.04 

.04  .01 

.00 

Retention  Cognitions 

n  =  928  - 

943 

n  =  943  -  959 

Common  TAPAS  Scales  (9) 

.13 

.19 

.06 

.01 

.12 

.18  .06 

.01 

Original  Can-Do 

.13 

.15 

.02 

.02 

.12 

.15  .03 

.02 

Original  Will-Do 

.13 

.17 

.04 

.03 

.12 

.16  .04 

.03 

Revised  Can-Do 

.13 

.13 

.00 

.00 

.12 

.13  .00 

.00 

Revised  Will-Do 

.13 

.14 

.01 

.00 

.12 

.14  .01 

.01 

New  Adaptation  a 

.13 

.13 

.00 

.00 

.12 

.12  .00 

.00 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT.  AFQT  +  TAPAS  =  Multiple 
correlation  ( R )  between  the  AFQT  +  selected  TAPAS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected 
TAPAS/TOPS  composite  scales  over  AFQT  to  the  regression  model  ([AFQT  +  TAPAS]  -  AFQT  Only).  Estimates  in  bold  were  statistically  significant,/?  <  .05 
(one-tailed).  Note  that  significance  tests  cannot  be  applied  to  the  adjusted  A R. 

a  Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Table  5.7.  Incremental  Validity  Estimates  for  the  TAPAS  overAFQT  for  Predicting  In-Unit  Physical  Fitness  and  Overall 
Performance  Criteria  by  Education  Tier 


Tier  1 

Tier  1+2  (Combined) 

In- Unit  Criterion  Measure  /  Model 

AFQT  R 

AFQT  + 
TAPAS  R 

A R 

Adj  A R 

AFQT  R 

AFQT  +  A„ 

TAPAS  R 

Adj  A R 

Physical  Fitness 

n  =  908  - 

923 

n  =  923  -  939 

Common  TAPAS  Scales  (9) 

.02 

.28 

.26 

.25 

.03 

.29 

.26 

.25 

Original  Can-Do 

.02 

.03 

.01 

.00 

.03 

.03 

.00 

.00 

Original  Will-Do 

.02 

.09 

.07 

.07 

.03 

.10 

.07 

.07 

Revised  Can-Do 

.02 

.04 

.02 

.00 

.03 

.04 

.01 

.00 

Revised  Will-Do 

.02 

.21 

.19 

.20 

.03 

.22 

.19 

.21 

New  Adaptation  a 

.02 

.14 

.12 

.13 

.03 

.15 

.12 

.13 

Overall  Performance 

n  =  654  - 

664 

n  =  664  -  674 

Common  TAPAS  Scales  (9) 

.11 

.18 

.07 

.00 

.10 

.18 

.08 

.01 

Original  Can-Do 

.11 

.12 

.00 

.00 

.10 

.10 

.00 

.00 

Original  Will-Do 

.11 

.15 

.03 

.03 

.10 

.14 

.04 

.03 

Revised  Can-Do 

.11 

.12 

.00 

.00 

.10 

.10 

.00 

.00 

Revised  Will-Do 

.11 

.15 

.04 

.04 

.10 

.15 

.05 

.04 

New  Adaptation  a 

.11 

.12 

.01 

.00 

.10 

.11 

.01 

.00 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT.  AFQT  +  TAPAS  =  Multiple 
correlation  ( R )  between  the  AFQT  +  selected  TAPAS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected 
TAPAS/TOPS  composite  scales  over  AFQT  to  the  regression  model  ([AFQT  +  TAPAS]  -  AFQT  Only).  Estimates  in  bold  were  statistically  significant,/?  <  .05 
(one-tailed).  Note  that  significance  tests  cannot  be  applied  to  the  adjusted  A R. 


Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Table  5.8.  Incremental  Validity  Estimates  for  the  TAPAS  over  AFQT  for  Predicting  Cumulative  Attrition  through  30  Months  of 
Service  by  Education  Tier 


Tier  1 

Tier  2 

Tier  1 

+  2 

Attrition  /  Model 

AFQT 

AFQT  + 

A  R 

Adj 

AFQT 

AFQT  + 

AR 

Adj 

AFQT 

AFQT  + 

AR 

Adj 

R 

TAPAS  R 

AR 

R 

TAPAS  R 

AR 

R 

TAPAS  R 

AR 

6-Month 

n  =  54,699- 

57,903 

n  =  1,564- 

1,768 

n  =  56,263 

-59,671 

Common  TAPAS  Scales  (9) 

.06 

.10 

.04 

.04 

.02 

.08 

.06 

.01 

.06 

.09 

.04 

.04 

Original  Can-Do 

.06 

.06 

.00 

.00 

.00 

.01 

.01 

.00 

.06 

.06 

.00 

.00 

Original  Will-Do 

.06 

.06 

.00 

.00 

.02 

.02 

.00 

.00 

.06 

.06 

.00 

.00 

Revised  Can-Do 

.06 

.06 

.00 

.00 

.00 

.01 

.01 

.00 

.06 

.06 

.00 

.00 

Revised  Will-Do 

.06 

.08 

.02 

.02 

.02 

.03 

.01 

.00 

.06 

.08 

.02 

.02 

New  Adaptation  :l 

.06 

.08 

.02 

.02 

.00 

.01 

.01 

.00 

.06 

.08 

.02 

.02 

12-Month 

n  =  41,512- 

41,672 

n  =  778  - 

783 

n  =  42,290 

-  42,455 

Common  TAPAS  Scales  (9) 

.06 

.10 

.05 

.04 

.06 

.11 

.05 

.00 

.06 

.10 

.05 

.04 

Original  Can-Do 

.06 

.06 

.00 

.00 

.05 

.06 

.00 

.00 

.06 

.06 

.00 

.00 

Original  Will-Do 

.06 

.06 

.01 

.00 

.06 

.06 

.00 

.00 

.06 

.06 

.00 

.00 

Revised  Can-Do 

.06 

.06 

.00 

.00 

.05 

.06 

.00 

.00 

.06 

.06 

.00 

.00 

Revised  Will-Do 

.06 

.09 

.03 

.03 

.06 

.07 

.01 

.00 

.06 

.09 

.03 

.03 

New  Adaptation  :l 

.06 

.09 

.03 

.03 

.05 

.05 

.00 

.00 

.06 

.09 

.03 

.03 

24-Month 

n  =  15,332  - 

15,385 

R 

II 

R 

77 

n  =  15,405 

- 15,462 

Common  TAPAS  Scales  (9) 

.10 

.14 

.03 

.03 

.18 

.48 

.30 

.16 

.10 

.14 

.03 

.03 

Original  Can-Do 

.10 

.11 

.00 

.00 

.14 

.14 

.00 

.00 

.10 

.10 

.00 

.00 

Original  Will-Do 

.10 

.11 

.01 

.01 

.19 

.28 

.09 

.07 

.10 

.11 

.01 

.01 

Revised  Can-Do 

.10 

.10 

.00 

.00 

.14 

.18 

.05 

.00 

.10 

.10 

.00 

.00 

Revised  Will-Do 

.10 

.13 

.02 

.02 

.19 

.28 

.09 

.07 

.10 

.12 

.02 

.02 

New  Adaptation  :l 

.10 

.13 

.03 

.03 

.14 

.29 

.15 

.16 

.10 

.13 

.03 

.03 

30-Month 

n  =  4,454  - 

4,502 

n  =  68  - 

72 

n  =  4,522 

-4,574 

Common  TAPAS  Scales  (9) 

.21 

.24 

.03 

.03 

.24 

.48 

.24 

.07 

.21 

.24 

.03 

.02 

Original  Can-Do 

.21 

.21 

.00 

.00 

.20 

.20 

.00 

.00 

.21 

.21 

.00 

.00 

Original  Will-Do 

.21 

.21 

.00 

.00 

.25 

.31 

.06 

.03 

.21 

.21 

.00 

.00 

Revised  Can-Do 

.21 

.21 

.00 

.00 

.20 

.22 

.02 

.00 

.21 

.21 

.00 

.00 

Revised  Will-Do 

.21 

.22 

.01 

.01 

.25 

.35 

.10 

.08 

.21 

.22 

.01 

.01 

New  Adaptation  :l 

.21 

.22 

.02 

.02 

.20 

.35 

.15 

.14 

.21 

.22 

.01 

.01 

Note.  The  number  in  parentheses  after  the  model  title  refers  to  the  number  of  TAPAS-related  scores  in  the  model  and  excludes  AFQT.  AFQT  +  TAPAS  =  Multiple  correlation  ( R )  between 
the  AFQT  +  selected  TAP  AS/TOPS  composite  scales  with  the  targeted  criterion  measure.  A R  =  Increment  in  R  from  adding  the  selected  TAPAS  scores  over  AFQT  to  the  regression  model 
([AFQT  +  TAPAS] — AFQT  Only).  Adj  AR=  the  AR  after  adjusting  for  shrinkage.  Estimates  in  bold  were  statistically  significant,  p  <  .05  (one-tailed).  Note  that  significance  tests  cannot  be 
applied  to  the  adjusted  AR. 

a  Because  the  Adaptation  composite  is  based  on  a  subset  of  the  data,  these  results  may  underestimate  its  incremental  validity. 


Summary 


This  chapter  summarized  results  from  the  sixth  cycle  of  the  evaluation  of  criterion-related  validity  in 
the  TOPS  IOT&E.  Overall  the  existing  and  new  TOPS  composites  demonstrated  modest  incremental 
validity  over  the  AFQT  in  predicting  first-term  Soldier  perfonnance  and  retention.  Incremental 
validity  estimates  (adjusted  for  shrinkage)  were  consistently  .04  or  less,  on  average,  for  the  original 
and  revised  Can-Do  and  Will-Do  composites  over  the  AFQT  used  alone.  In  contrast,  the  revised 
Will-Do  composite  was  associated  with  the  greatest  incremental  validity  gains,  on  average, 
compared  to  other  TOPS  composites.  This  was  especially  true  for  the  prediction  of  physical  fitness, 
adjustment  to  Army  life,  commitment  and  fit,  and  discipline.  None  of  the  TOPS  composites 
demonstrated  utility  in  incrementing  the  AFQT  in  the  prediction  of  attrition  up  to  30  months  in 
service. 
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CHAPTER  6:  INFORMATION/COMMUNICATIONS  Technology  LITERACY  TEST 

EVALUATION 

D.  Matthew  Trippe,  Thomas  Kiger,  and  Bethany  Bynum  (HumRRO) 


Background  on  Development  and  Validation  of  the  ICTL  Test 

One  of  the  Department  of  Defense  (DoD)  ASVAB  review  panel’s  recommendations  stated  that 
“ research  should  be  conducted  to  develop  and  evaluate  a  test  of  information  and 
communications  technology  literacy  (ICTL).  The  efficacy  of  coaching  and  item  familiarity,  as 
well  as  the  feasibility  of  creating  multiple  forms,  should  be  examined  in  conjunction  with  test 
development”  (Drasgow  et  ah,  2006,  p.  26).  Toward  that  end,  the  U.S.  Air  Force  assumed 
responsibility  as  the  lead  organization  in  development  of  an  ICTL  test  which  could  potentially  be 
added  to  the  ASVAB.  The  Air  Force  retained  HumRRO  to  develop  and  validate  the  ICTL  test. 

Development  of  the  ICTL  test  began  with  the  creation  of  a  taxonomy  of  knowledge,  skills,  abilities, 
and  other  characteristics  (KSAOs)  required  for  successful  perfonnance  in  cyber/IT  occupations.  The 
final  content  blueprint  focused  on  the  four  broad  knowledge  content  areas  of  (a)  Networking  and 
Communications,  (b)  Computer  Operations,  (c)  Security  and  Compliance,  and  (d)  Software 
Programming  and  Web  Development.  Once  the  content  to  be  measured  was  defined,  attention 
turned  to  identifying  item  types  and  measurement  methods.  Although  several  item  types  were 
considered,  there  were  a  number  of  constraints  related  to  testing  time  and  compatibility  with  the 
ASVAB  platfonn  that  virtually  dictated  a  multiple-choice  fonnat.  Multiple  rounds  of  pilot  testing 
and  item  review  ultimately  produced  two  parallel  29-item  operational  ICTL  test  fonns. 

Initial  evidence  of  the  ICTL’s  predictive  validity  was  obtained  from  seven  Air  Force  technical 
training  schools  and  three  Navy  schools.  Promising  results  led  the  Air  Force  to  develop 
operational  cut  scores  to  be  used  in  a  compensatory  fashion  with  existing  ASVAB  composites  to 
select  airmen  for  enlisted  cyber-related  occupations. 

A  parallel  validation  effort  is  currently  underway  involving  cyber-related  MOS  in  the  Army.  The 
Anny’s  Signal  Center  of  Excellence  requested  that  ARI  assist  in  the  development  of  a 
methodology  to  improve  the  trainee  selection  process.  ARI  designed  a  research  study  to  validate 
the  ICTL  test  in  predicting  trainee  perfonnance  in  Infonnation  Systems  Operator-Analyst  (25B) 
and  Nodal  Network  Systems  Operator-Maintainer  (25N)  MOS.  MOS-specific  IMT  criterion  data 
are  being  collected  from  Soldiers  in  these  MOS,  though  sample  sizes  are  low  (especially  for  25N) 
since  data  collection  did  not  start  until  2012.  Initial  indications  are  that  the  ICTL  test  is  a 
significant  predictor  of  training  perfonnance,  academic  retention,  and  perceptions  of  MOS  fit. 
Although  there  is  some  overlap  between  the  Signal  Center  validation  study  and  the  TOPS 
validation  analyses  described  in  this  chapter,  the  TOPS  analyses  introduce  additional  Soldiers  in 
MOS  beyond  those  trained  in  the  Signal  Center.  This  allows  the  opportunity  to  evaluate  convergent 
and  discriminant  validity,  and  expands  the  breadth  of  validity  evidence  for  this  relatively  new 
assessment.  10 


10  The  ICTL  test  can  be  said  to  demonstrate  convergent  and  discriminant  validity  if  it  is  more  predictive  in 
occupations  with  a  higher  degree  of  conceptual  overlap  (i.e.,  25B)  and  less  predictive  in  MOS  with  a  lower  degree 
of  conceptual  overlap  (e.g.,  1  IB). 
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ICTL  Validation  Sample 


Soldiers  in  the  ICTL  Validation  Sample  were  administered  the  ICTL  at  MEPS  from  May  2011  to 
January  2012  as  part  of  the  original  ICTL  validation  research  conducted  across  multiple 
Services.  This  sample  included  52,708  Service  applicants,  of  which  14,553  were  Army  Regular, 
6,156  were  Army  National  Guard,  and  3,278  were  Anny  Reserve  applicants.  The  vast  majority 
(93%)  of  these  applicants  were  linked  to  a  TAPAS  record  to  become  part  of  the  TOPS  Applicant 
Sample.  The  ICTL  Validation  Sample  was  created  by  limiting  the  TOPS  validation  samples 
(Administrative  and  IMT)  described  in  Chapter  2  to  Soldiers  with  valid  ICTL  scores.  We  further 
limited  the  ICTL  Validation  Sample  to  the  1,758  Soldiers  in  five  target  MOS  (1  IB,  25B,  3  IB, 
68W,  and  88M)  with  sample  sizes  greater  than  50.  To  clean  the  ICTL  test  data,  we  removed 
individuals  who  responded  to  fewer  than  25  ICTL  items  or  who  completed  the  assessment  in  less 
than  3  minutes.  Three  minutes  is  a  rather  liberal  criterion  used  to  remove  only  the  most  extreme 
outliers  and  is  merely  one  of  many  data  screens  applied.  In  addition,  we  removed  any 
respondents  who  scored  at  or  below  chance  on  the  ICTL  test  and  also  scored  above  average  on 
the  AFQT.  All  data  screens  combined  resulted  in  the  removal  of  only  288  applicants,  which  is 
likely  a  reflection  of  the  fact  that  the  ICTL  test  was  presented  seamlessly  with  operational  pre¬ 
enlistment  assessments.  Table  6.1  contains  the  demographic  characteristics  of  the  TOPS  ICTL 
validation  sample. 

Table  6. 1 .  Background  and  Demographic  Characteristics  of  the  TOPS  ICTL  Validation  Sample 


Characteristic 

ICTL  Validation 

n  = 

1,758 

n 

% 

Component 

Regular 

980 

55.7 

ARNG 

519 

29.5 

USAR 

259 

14.7 

Education  Tier 

Tier  1 

1,661 

94.5 

Tier  2 

97 

5.5 

Military’  Occupational  Specialty 

1  IB/1 1C/1 1X/18X 

513 

29.2 

25B 

469 

26.7 

31B 

283 

16.1 

68W 

221 

12.6 

88M 

272 

15.5 

AFQT  Category’ 

I 

145 

8.3 

II 

682 

38.8 

IIIA 

408 

23.2 

1 1  IB 

450 

26.5 

IV 

73 

4.2 

Gender 

Female 

263 

15.1 

Male 

1,480 

79.4 
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Table  6.1.  (Continued) 


Characteristic 

ICTL  Validation 

n  = 

1,758 

n 

% 

Race 

African  American 

274 

15.6 

American  Indian 

16 

0.9 

Asian 

64 

3.6 

Hawaiian/Pacific  Islander 

10 

0.6 

Caucasian 

1,299 

73.9 

Multiple 

4 

0.2 

Declined  to  Answer 

91 

5.2 

Ethnicity 

Hispanic/Latino 

204 

11.6 

Not  Elispanic 

1,478 

84.1 

Declined  to  Answer 

76 

4.3 

Table  6.2  presents  ICTL  test  scores  by  MOS  in  the  scaled  reporting  metric.  ICTL  scaled  scores 
are  an  IRT-based  ability  estimate  that  has  been  placed  on  an  adjusted  /-score  scale.  A  standard  t- 
score  distribution  has  a  mean  of  50  and  standard  deviation  of  10.  The  ICTL  reporting  metric  has 
been  adjusted  such  that  the  standard  distribution  would  be  expected  in  the  youth  population 
(Profile  of  American  Youth  [PAY97]  sample;  DMDC,  2003).  Table  6.2  reveals  that,  on  average, 
Soldiers  in  the  68W  MOS  have  the  highest  ICTL  scores,  followed  by  25B,  1  IB,  3  IB  and  88M. 
The  ICTL  test  was  designed  to  predict  training  perfonnance  in  cyber  related  occupations  like 
25B,  but  like  most  cognitive  assessments,  reflects  general  mental  ability  to  some  degree. 


Table  6.2.  ICTL  Scaled  Scores  by  MOS 


MOS 

n 

Min 

Max 

M 

SD 

11B 

513 

30.0 

80.0 

52.9 

8.5 

25B 

469 

33.0 

77.0 

54.8 

8.3 

31B 

283 

34.0 

72.0 

52.3 

7.3 

68W 

221 

38.0 

77.0 

57.9 

7.9 

88M 

272 

31.0 

73.0 

51.4 

8.1 

Full  Sample 

1,758 

30.0 

80.0 

53.7 

8.3 

Table  6.3  contains  subgroup  mean  comparisons  for  selected  groups.  Gender,  race,  and  ethnicity 
groups  were  chosen  to  be  consistent  with  designations  used  by  the  ASVAB  testing  program 
(Defense  Manpower  Data  Center,  2011).  Standardized  mean  difference  values  reveal  statistically 
significant  differences  between  minority  and  majority  gender,  race,  and  ethnicity  groups  such 
that  the  majority  outperforms  the  minority  group  in  all  comparisons.  The  mean  difference 
between  education  tiers  is  not  statistically  significant.  Standardized  mean  difference  values  are 
comparable  to  those  observed  in  the  original  ICTL  validation  sample.  It  is  important  to  note  that 
ICTL  subgroup  differences  in  gender,  race,  and  ethnicity  groups  are  smaller  than  those  observed 
for  other  ASVAB  technical  tests  (Trippe  &  Russell,  201 1). 11 


11  ASVAB  technical  tests  include  Auto-Shop  (AS),  General  Science  (GS),  Electronics  Information  (El)  and 
Mechanical  Comprehension  (MC). 
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Table  6.3.  ICTL  Scaled  Scores  by  Subgroup 


Minority 

Majority 

Subgroup  Comparison 

n 

M 

SD 

n 

M 

SD 

d 

Female  -  Male 

263 

51.2 

6.6 

1480 

54.1 

8.6 

0.35 

Black  -  White 

274 

50.4 

7.3 

1106 

54.6 

8.5 

0.51 

Hispanic  -  Non-Hispanic  White 

204 

51.3 

7.3 

1106 

54.6 

8.5 

0.40 

Ed  Tier  2  -  Ed  Tier  1 

97 

55.8 

7.7 

1661 

53.6 

8.4 

-0.27 

Note,  d  =  Cohen's  d,  positive  means  the  majority  group  mean  is  larger  than  the  minority  group  mean.  Bolded  values 
indicate  significant  /-tests  (p  <  .05,  two-tailed). 


ICTL  Validation  Analyses 

The  ICTL  test  is  a  cognitive  measure  designed  in  the  mold  of  ASVAB  tests  that  should 
theoretically  have  the  strongest  relationship  with  cyber-specific  task-  or  knowledge-based 
performance  outcomes  such  as  course  grades,  training  restart,  and  MOS-specific  job 
knowledge/perfonnance  (i.e.,  “Can  Do”  criteria).  This  will  be  somewhat  difficult  to  assess  in  the 
purest  sense  because  there  are  no  MOS-specific  JKT  data  for  the  only  cyber  related  target  MOS 
(25B)  in  the  ICTL  analysis  database. 

The  ICTL  test  may  also  function  well  as  an  indirect  indicator  of  select  fit-  or  motivation-based 
performance  outcomes  (i.e.,  “Will  Do”  criteria)  in  relevant  occupations.  Similar  to  ASVAB 
technical  subtests,  the  ICTL  measure  is  an  information  test.  Infonnation  tests  were  among  the 
most  successful  and  most  highly  valid  printed  classification  tests  created  by  the  Anny  Air 
Force’s  (AAF)  Aviation  Psychology  Program  during  World  War  II.  Guilford  and  Lacey  (1947) 
described  the  logic  of  information  tests  as  follows: 

It  is  becoming  recognized  more  and  more  that  what  a  person  knows  or  does  not 
know  can  be  used  to  reveal  a  number  of  things  concerning  his  personal 
background.  Since  he  is  to  a  large  extent  a  product  of  his  personal  experience, 
and  since  what  he  is  bodes  good  or  ill  concerning  his  future  status  in  one  respect 
or  another,  knowledge  scores  promise  to  have  predictive  value,  (p.  341) 

The  key  notion  is  that  information  tests  are  thought  to  be  indirect  measures  of  interest, 
motivation,  and  skill  in  a  particular  area. 

Table  6.4  contains  the  bivariate  correlations  between  ICTL  scores  and  available  predictor  and 
criterion  variables.  Among  the  predictors,  ICTL  scores  have  the  strongest  relationship  with  the 
General  Science  (GS)  and  Electronics  Infonnation  (El)  subtests  as  well  as  the  Verbal  (VE) 
composite.  These  relationships  are  consistent  with  those  observed  in  the  larger  Service  applicant 
sample  (Trippe  &  Russell,  2011).  In  addition  to  sharing  some  conceptual  overlap  with  GS  and  El 
tests,  the  ICTL  test  has  a  verbal  load  such  that  sufficient  language  ability  is  required  to  access  the 
content.  The  general  positive  association  between  ICTL  and  all  cognitively  based  measures  (i.e., 
ASVAB  tests,  Knowledge  &  Skill  criterion  composite)  is  also  noteworthy. 
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Table  6.4.  Predictor/Criterion  Relationships  with  ICTL 


Predictor/Criterion 

n 

r 

Predictor/Criterion 

n 

r 

AFQT 

1758 

.62 

12-Month  Attrition 

417 

-.03 

ASVAB  -  GS 

1758 

.56 

18 -Month  Attrition 

39 

.20 

ASVAB  -  AR 

1758 

.45 

Army  Life  Adjustment 

1493 

.00 

ASVAB  -  WK 

1758 

.57 

Overall  Performance 

365 

.01 

ASVAB  -  PC 

1758 

.48 

Physical  Fitness 

1422 

-.06 

ASVAB  -  MK 

1758 

.36 

Commitment  &  Fit 

1430 

-.02 

ASVAB  -  El 

1758 

.52 

MOS  Fit 

1493 

-.01 

ASVAB  -  AS 

1758 

.31 

Retention  Cognitions 

1430 

-.11 

ASVAB  -  MC 

1758 

.48 

Knowledge  &  Skill 

1127 

.39 

ASVAB -  AO 

1720 

.31 

Training  Restart 

1550 

-.02 

ASVAB  -  VE 

1758 

.59 

Disciplinary  Incidents 

1429 

.02 

6-Month  Attrition 

932 

.02 

Note.  Bolded  correlations  are  significant  (p  <  .05;  two-tailed). 


ICTL  scores  have  statistically  significant  relationships  with  the  Knowledge  &  Skill,  Retention 
Cognitions,  and  Physical  Fitness  criterion  scores.  The  latter  two  relationships  are  negative, 
suggesting  those  with  higher  ICTL  aptitude  feel  less  strongly  about  making  the  Army  a  career 
and  are  slightly  less  physically  fit.  The  positive  relationship  with  the  Knowledge  &  Skill  and 
Retention  Cognitions  composites  is  consistent  with  the  expectation  of  ICTL  as  a  predictor  of  Can 
Do  performance.  The  relationship  between  ICTL  and  the  Commitment  and  Fit  composite  is 
effectively  zero.  The  ICTL  test  is  not  expected  to  be  a  general  predictor  of  Will  Do  outcomes 
across  MOS,  but  rather  should  only  function  as  such  for  cyber  related  occupations. 

Table  6.5  contains  the  correlations  between  ICTL  and  available  criteria  by  MOS.  ICTL  scores 
have  the  strongest  relationship  with  the  Knowledge  and  Skill  criterion  composite.  Correlations 
are  statistically  significant  for  all  MOS  and  do  not  exhibit  evidence  of  discriminant  validity.  That 
is,  the  relationship  between  ICTL  and  Knowledge  and  Skill  scores  does  not  appear  to  vary  by 
MOS  such  that  the  ICTL  test  is  more  strongly  related  to  perfonnance  for  the  25B  MOS  (i.e.,  the 
one  cyber  related  occupation  in  the  target  MOS)  than  the  remaining  MOS.  Recall,  however,  that 
the  Knowledge  and  Skill  composite  combines  MOS-specific  and  WTBD  JKTs  or  AIT  course 
grades  if  available.  AIT  course  grades,  but  no  MOS-specific  JKT  data,  are  available  for  25B,  but 
are  not  available  in  substantial  numbers  for  Soldiers  enlisted  in  the  other  MOS.  With  these 
limitations  in  mind,  we  nevertheless  attempted  to  further  focus  on  discriminant  validity  for  the 
prediction  of  cyber  specific  knowledge  and  skill  outcomes.  To  do  so,  we  examined  the 
relationship  between  ICTL  scores  and  the  available  indicators  of  cyber-specific  Can  Do 
performance.  Table  6.5  reports  correlations  between  ICTL  scores  and  the  MOS-specific  JKT  or 
AIT  course  grades,  as  available.  A  simple  structure  pattern  of  discriminant  validity  does  not 
emerge,  but  criterion  type  is  confounded  with  MOS  in  this  sample.  The  ICTL  test  is  a  cognitive 
measure  that  will  have  a  moderate  to  strong  relationship  with  other  measures  that  share  a  general 
mental  ability  component.  The  absence  of  a  common  cyber-specific  criterion  measure  and 
presence  of  a  generally  high  degree  of  association  among  cognitively  based  assessments  combine 
to  make  differential  validity  difficult  to  evaluate  in  this  perfonnance  domain. 


49 


Next,  the  extent  to  which  the  ICTL  could  be  used  as  an  indicator  of  fit  was  examined.  As  indicated  in 
Table  6.5,  ICTL  scores  have  a  relatively  strong  relationship  with  perceptions  of  MOS  fit.  ICTL 
scores  also  have  a  significant  positive  relationship  with  MOS  Fit  in  25B  and  significant  negative 
relationships  with  MOS  Fit  in  1  IB  and  88M.  The  relationship  is  not  significant  in  3  IB  and  68W. 

This  pattern  of  differential  validity  suggests  that  ICTL  is  a  useful  and  valid  predictor  of  MOS  fit  and 
thus  potentially  valid  as  a  classification  tool  in  this  regard.  No  other  patterns  of  differential  validity 
emerge  in  Table  6.5.  This  includes  the  Training  Restart  score,  in  which  a  differential  validity 
structure  may  be  expected.  Although  not  a  very  large  effect,  ICTL  scores  have  a  significant  negative 
relationship  with  the  Physical  Fitness  score  in  the  25B  and  68W  MOS.  ICTL  scores  also  have  a 
significant  negative  relationship  with  the  Retention  Cognitions  in  1  IB  and  25B. 

Table  6.6  contains  incremental  validity  of  ICTL  scores  beyond  AFQT  by  MOS.  ICTL  scores 
provide  statistically  significant  incremental  validity  beyond  AFQT  in  predicting  Knowledge  and 
Skill  scores  in  the  25B,  3  IB,  and  88M  MOS.  The  same  pattern  of  significance  emerges  in  the 
incremental  validity  analysis  of  available  MOS-specific  JKTs  and  AIT  course  grades.  Although 
25B  is  the  only  MOS  for  which  the  ICTL  test  is  directly  relevant,  the  cognitive  elements  shared 
by  predictor  and  criterion  assessments  is  evident  in  these  MOS.  That  is,  the  ICTL  test  and  the 
Knowledge  &  Skill  composite  (as  well  as  the  individual  assessments  that  comprise  the 
composite)  have  a  known  general  mental  ability  component.  It  is  difficult  to  separate  the  general 
and  specific  dimensions  of  each  predictor-criterion  relationship. 

The  25B  MOS  is  the  only  occupation  for  which  ICTL  scores  provide  statistically  significant 
incremental  validity  in  predicting  Retention  Cognitions.  Unfortunately  ICTL  scores  have  a 
negative  relationship  with  this  score,  suggesting  that  those  of  higher  aptitude  have  a  weaker  desire 
to  remain  in  their  occupation.  These  higher  aptitude  individuals  may  be  more  likely  than  lower 
aptitude  individuals  to  have  attractive  job  opportunities  outside  of  military  Service.  ICTL  scores 
also  provide  significant  incremental  validity  in  predicting  MOS  Fit  in  the  25B  and  1  IB  MOS.  The 
relationship  between  ICTL  and  MOS  Fit  is  negative  in  the  1  IB  MOS,  suggesting  those  of  higher 
ICTL  aptitude  fit  less  well  in  the  1  IB  MOS.  Conversely,  the  relationship  between  ICTL  and  MOS 
Fit  is  positive  in  25B. 
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Table  6.5.  ICTL  Relationships  with  Outcomes  by  MOS 


Criteria 

1  IB 

n  r 

25B 

n  r 

31B 

n  r 

68W 

n  r 

88M 

n  r 

6-Month  Attrition 

397 

.07 

- 

- 

138 

.01 

119 

- 

~ 

- 

12 -Month  Attrition 

235 

.02 

- 

- 

50 

-.12 

93 

.05 

- 

- 

Army  Life  Adjustment 

496 

-.04 

228 

.07 

281 

.06 

217 

.09 

271 

-.04 

Overall  Performance 

143 

.05 

- 

- 

104 

.05 

118 

-.04 

- 

- 

Physical  Fitness 

487 

-.03 

192 

-.15 

273 

-.07 

208 

-.16 

262 

.00 

Commitment  &  Fit 

488 

-.08 

194 

.14 

274 

.01 

211 

.06 

263 

-.06 

MOS  Fit 

496 

-.15 

228 

.29 

281 

-.07 

217 

.04 

271 

-.24 

Retention  Cognitions 

488 

-.12 

194 

-.27 

274 

.04 

211 

.02 

263 

-.09 

Knowledge  &  Skill 

376 

.39 

105 

.48 

252 

.47 

187 

.25 

207 

.51 

MOS-Specific  JKT 

389 

.35 

- 

- 

259 

.43 

195 

.15 

213 

.39 

AIT  Course  Grade 

- 

- 

296 

.45 

- 

- 

- 

- 

- 

- 

Training  Restart 

502 

.02 

295 

-.06 

276 

-.07 

213 

-.01 

264 

-.09 

Disciplinary  Incidents 

488 

-.01 

194 

.06 

273 

-.03 

211 

.07 

263 

.02 

Note.  Bolded  correlations  are  significant  (p  <  .05;  two-tailed).  Missing  cells  indicate  insufficient  data  ( n  <  50)  or  no  variance  in  the  outcome  (i.e.,  no  attrition  at  6 
months). 


Table  6.6.  Incremental  Validity  ofICTL  over  AFQT 


1  IB 

25B 

31B 

AFQT 

AFQT+ 

ICTL 

A R 

AFQT 

AFQT+ 

ICTL 

A R 

AFQT 

AFQT+ 

ICTL 

A R 

n 

r  (rDb) 

R  {rDh) 

A  rDb 

n 

R  Vah) 

R  Vnb) 

•O 

<1 

n 

R  (Nh) 

R  Vnb) 

A  rvh 

6-Month  Attrition 

396 

.07 

.08 

.00 

- 

- 

- 

- 

137 

.08 

.10 

.01 

12 -Month  Attrition 

234 

.01 

.02 

.01 

- 

- 

- 

- 

49 

.05 

.19 

.14 

Army  Life  Adjustment 

495 

.09 

.16 

.07 

227 

.08 

.09 

.00 

280 

.01 

.07 

.06 

Overall  Performance 

142 

.21 

.26 

.04 

- 

- 

- 

- 

103 

.03 

.05 

.02 

Physical  Fitness 

486 

.16 

.25 

.09 

191 

.14 

.16 

.02 

272 

.02 

.07 

.05 

Commitment  &  Fit 

487 

.02 

.10 

.08 

193 

.09 

.14 

.05 

273 

.05 

.06 

.02 

MOS  Fit 

496 

.05 

.17 

.12 

228 

.18 

.29 

.11 

283 

.14 

.14 

.00 

Retention  Cognitions 

487 

.20 

.20 

.00 

193 

.26 

.30 

.04 

273 

.06 

.10 

.05 

Knowledge  &  Skill 

375 

.53 

.54 

.00 

104 

.55 

.58 

.03 

251 

.51 

.55 

.04 

MOS-Specific  JKT 

389 

.45 

.46 

.01 

- 

- 

- 

- 

259 

.47 

.51 

.04 

AIT  Course  Grade 

- 

- 

- 

- 

296 

.42 

.49 

.08 

- 

- 

- 

- 

Training  Restart 

501 

.04 

.07 

.04 

294 

.02 

.07 

.05 

275 

.06 

.07 

.01 

Disciplinary  Incidents 

487 

.08 

.10 

.02 

193 

.12 

.12 

.00 

272 

.03 

.03 

.00 

n 

68W 

AFQT  A^+ 
R  ( rvb )  R  (rDb) 

A R 

A  r„b 

n 

88M 

AFQT  AW+ 
R  ( rvb )  R  (rDb) 

A R 

A  rpb 

6-Month  Attrition 

~ 

- 

- 

- 

- 

- 

- 

- 

12 -Month  Attrition 

92 

.04 

.09 

.05 

- 

- 

- 

- 

Army  Life  Adjustment 

216 

.12 

.13 

.01 

270 

.07 

.07 

.00 

Overall  Performance  Ratings 

117 

.04 

.05 

.01 

- 

- 

- 

- 

Physical  Fitness 

207 

.06 

.22 

.16 

261 

.01 

.02 

.01 

Commitment  and  Fit 

210 

.07 

.07 

.00 

262 

.13 

.14 

.00 

MOS  Fit 

217 

.09 

.09 

.00 

271 

.28 

.29 

.01 

Retention  Cognitions 

210 

.04 

.06 

.02 

262 

.15 

.15 

.00 

Knowledge  &  Skill 

186 

.32 

.34 

.02 

206 

.47 

.54 

.07 

MOS  Specific  JKT 

195 

.26 

.26 

.00 

213 

.40 

.43 

.03 

AIT  Course  Grade 

- 

- 

- 

- 

- 

- 

- 

- 

Training  Restart 

212 

.20 

.22 

.02 

263 

.08 

.09 

.02 

Disciplinary  Incidents 

210 

.11 

.18 

.07 

262 

.02 

.05 

.03 

Note.  Bolded  correlations  are  significant  (p  <  .05;  two-tailed).  Missing  cells  indicate  insufficient  data  (n  <  50). 


Summary  and  Discussion 


Results  of  the  validity  analyses  presented  here  suggest  that  the  ICTL  test  is  a  valid  predictor  of 
both  Can  Do  and  Will  Do  performance  dimensions.  ICTL  scores  have  an  appreciable  positive 
relationship  with  (a)  the  Knowledge  &  Skill  composite  and  (b)  available  MOS-specific  JKTs  and 
AIT  course  grades  in  all  MOS.  ICTL  scores  provide  significant  incremental  validity  in  predicting 
this  Can  Do  indicator  for  three  out  of  five  MOS.  The  pattern  of  validity  and  incremental  validity 
results  in  predicting  Can  Do  outcomes  is  not  readily  interpretable  as  demonstrating  discriminant 
validity.  That  is,  ICTL  scores  do  not  appear  to  necessarily  be  better  or  stronger  predictors  of  Can 
Do  performance  for  25B  than  the  other  MOS.  Evaluation  convergent  and  discriminant  validity 
are  nevertheless  complicated  by  (a)  the  absence  of  a  common  occupation-specific  criterion 
measure  and  (b)  the  high  level  of  general  positive  association  among  cognitively  based  predictor 
and  criterion  measures.  Complicating  matters  even  further  is  the  fact  that  25B  is  not  the  only 
MOS  for  which  technology  is  relevant.  Although  25B  is  the  only  MOS  for  which  criterion 
assessments  will  have  a  direct  cyber-specific  component,  general  performance  in  virtually  all 
MOS  will  involve  information  and  communications  technology  literacy.  Those  who  are  facile 
with  technology  will  have  an  edge  over  those  who  do  not  in  nearly  any  occupation.  Thus,  the 
finding  that  the  ICTL  showed  positive  relationships  across  MOS  is  not  altogether  surprising. 

The  clearest  evidence  of  convergent  and  discriminant  validly  is  observed  in  the  pattern  of 
relationships  between  ICTL  scores  and  MOS  Fit.  ICTL  scores  have  a  significant  positive 
relationship  with  MOS  Fit  in  the  25B  MOS  and  either  no  relationship  or  a  significant  negative 
relationship  in  the  remaining  MOS.  As  noted  previously,  ICTF  scores  may  be  an  indicator  of 
interest  in  the  area  of  infonnation  technology  which  ultimately  leads  to  greater  perceived  fit. 

Overall,  the  ICTF  is  a  promising  and  useful  new  predictor.  It  proved  to  not  only  be  a  valid 
predictor  of  both  Can  Do  and  Will  Do  perfonnance  dimensions  but  also  displayed  incremental 
validity  over  the  AFQT  for  important  outcomes. 
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CHAPTER  7:  SUMMARY  AND  A  LOOK  AHEAD 


Deirdre  J.  Knapp  (HumRRO),  Kate  LaPort,  Tonia  S.  Heffner,  and  Leonard  A.  White  (ARI) 


Summary  of  the  TOPS  IOT&E  Method 

In  an  effort  to  expand  the  basis  on  which  applicants  are  evaluated  for  enlistment,  the  Anny  is 
conducting  an  initial  operational  test  and  evaluation  (IOT&E)  of  the  Tier  One  Performance 
Screen  (TOPS).  The  TOPS  assessments,  including  the  Tailored  Adaptive  Personality  Assessment 
System  (TAPAS),  the  Information/Communications  Technology  Literacy  (ICTL)  test,  and 
starting  in  FY2014,  the  Work  Preferences  Assessment  (WPA),  are  being  administered  to  non¬ 
prior  service  applicants  testing  at  MEPS  locations. 

To  evaluate  the  TAPAS,  ICTL,  and  WPA,  the  Army  is  collecting  training  criterion  data  on 
Soldiers  in  selected  MOS  as  they  complete  their  IMT.  The  criterion  measures  include  job 
knowledge  tests  (JKTs),  an  attitudinal  person-environment  fit  assessment  (the  Army  Life 
Questionnaire;  ALQ),  and  performance  rating  scales  (PRS)  completed  by  the  Soldiers’  cadre 
members.  Course  grades  and  completion  rates  are  obtained  from  administrative  records  for  all 
Soldiers,  regardless  of  MOS.  The  plan  is  to  construct  analysis  datasets  and  conduct  cumulative 
validation  analyses  at  6-month  intervals  throughout  the  IOT&E  period. 

Job  performance  data  are  also  being  collected  from  Soldiers  in  their  units  to  gather  data  on 
Soldiers  who  completed  the  TAPAS  (and  WPA  and  ICTL)  at  entry.  These  measures  again 
include  JKTs,  the  ALQ,  and  supervisor  ratings.  Finally,  the  separation  status  of  all  Soldiers  who 
took  the  TAPAS  at  entry  is  being  tracked  throughout  the  course  of  the  research. 

The  December  2012  data  file  includes  a  total  of  344,953  applicants  who  took  the  TAPAS.  Of 
these,  309,1 10  were  in  the  TOPS  Applicant  Sample.  The  Applicant  Sample  was  determined  by 
excluding  Education  Tier  3,  AFQT  Category  V,  and  prior  service  applicants  from  the  master  data 
file.  Of  that  Applicant  Sample,  141,170  (45.7%)  had  a  record  in  at  least  one  of  the  administrative 
criterion  data  sources;  17,670  had  IMT  data  collected  from  the  schoolhouse  and  1,053  had  in¬ 
unit  criterion  data. 

Data  from  the  JKTs,  PRS,  ALQ,  and  administrative  sources  were  combined  to  yield  an  array  of 
scores  representing  important  Soldier  outcomes.  In  general,  the  criterion  scores  exhibit 
acceptable  and  theoretically  consistent  psychometric  properties.  The  exception  to  this  is  the 
Anny-wide  and  MOS-specific  PRS,  which  continue  to  exhibit  low  interrater  reliability 
coefficients.  Results  involving  the  PRS  should  be  interpreted  with  caution. 
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Summary  of  Evaluation  Results  to  Date 


Evaluation  results  thus  far  suggest  that,  while  the  magnitude  of  the  predictive  validity  estimates 
are  not  as  large  as  those  found  in  the  experimental  EEEM  research  (Knapp  &  Heffner,  2010),  the 
TAPAS  holds  promise  for  new  Soldier  selection.  Results  of  the  incremental  validity  analyses 
indicate  that  the  TAPAS  predicts  important  first-term  criteria  over  and  above  the  AFQT, 
especially  measures  tapping  non-technical  aspects  of  Soldier  performance,  such  as  physical 
fitness,  adjustment  to  Army  life,  commitment  and  fit,  and  discipline.  The  revised  Will-Do  composite 
was  associated  with  the  greatest  incremental  validity  gains  compared  to  other  TOPS  composites. 

This  was  especially  true  for  the  prediction  of  physical  fitness.  None  of  the  TOPS  composites 
demonstrated  utility  in  incrementing  the  AFQT  in  the  prediction  of  attrition  up  to  30  months  in 
service.  Results  of  the  previously  reported  classification  analyses,  however,  indicated  that  the 
TAPAS  has  the  potential  to  enhance  matching  new  Soldiers  to  MOS,  particularly  for  minimizing 
attrition.  That  is,  while  the  TOPS  composites  did  not  show  global  utility  in  predicting  attrition, 
attrition  may  be  decreased  by  better  matching  Soldiers  with  MOS  using  the  TAPAS. 

Results  of  the  ICTL  validity  analyses  suggest  that  the  ICTL  test  is  a  valid  predictor  of  both  Can 
Do  and  Will  Do  performance  dimensions  across  both  cyber  and  other  MOS.  Specifically,  ICTL 
scores  have  an  appreciable  positive  relationship  with  (a)  the  Knowledge  &  Skill  composite  and 
(b)  available  MOS-specific  JKTs  and  AIT  course  grades  in  all  MOS.  ICTL  scores  provide 
significant  incremental  validity  in  predicting  this  Can  Do  indicator  for  three  out  of  five  MOS. 
Attempts  to  examine  discriminant  validity  evidence  is  complicated  by  the  lack  of  MOS-specific 
criterion  data  for  the  primary  cyber  MOS  (25B)  included  in  the  currently  available  database. 

Looking  Ahead 
Changes  to  Predictor  Measures 

In  FY2014,  a  third  series  of  new  adaptive  fonns  of  the  TAPAS  will  be  introduced  at  the  MEPS.  Each 
fonn  measures  13  dimensions.  All  three  13D  fonns  assess  the  same  10  core  dimensions,  plus  three  of 
seven  experimental  dimensions.  The  seven  experimental  dimensions  assessed  vary  by  version.  In 
total,  the  newer  versions  of  the  TAPAS  collectively  measure  17  dimensions.  The  experimental 
dimensions  will  be  evaluated  for  potential  use  in  revised  or  new  TOPS  composites,  once  sufficient 
data  are  available. 

Along  with  the  new  TAPAS  versions,  a  new  TOPS  screen  also  will  be  fielded  to  select  new 
Soldiers.  The  new  TOPS  screen  will  be  based  on  the  new  TOPS  composites  (Can-Do,  Will-Do, 
Adaptation).  The  new  TOPS  composites  incorporate  several  enhancements  over  the  existing 
composites  and  were  constructed  from  analyses  of  the  most  current  IOT&E  data.  Chapter  5 
summarized  the  predictive  validity  of  the  new  TOPS  composites,  compared  to  the  existing 
composites.  Overall,  the  new  Will-Do  and  Adaptation  composites  evidenced  higher  predictive 
validity,  on  average,  than  the  existing  composites,  although  the  magnitude  of  the  gains  varied  by 
criterion. 
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Analyses 


The  semi-annual  analyses  will  continue  to  include  basic  psychometric,  validation,  and 
incremental  validation  analyses.  In  coming  years,  however,  we  will  document  analyses  in  a 
single  annual  technical  report  rather  than  two  technical  reports  (interim  and  annual).  As  needed, 
we  will  examine  the  comparability  of  new  TAPAS  versions  to  prior  fonns  before  detennining  if 
the  data  can  be  combined  for  purposes  of  analysis.  We  also  will  try  an  alternative  approach  from 
prior  analysis  cycles  to  modeling  MOS  classification  outcomes  that  may  result  in  evaluation 
results  that  can  more  meaningfully  inform  policy  decisions. 

The  next  set  of  TOPS  evaluation  analyses  will  be  conducted  based  on  a  data  file  constructed  in 
June  2013.  We  will  continue  to  update  or  to  modify  our  analysis  plans  as  the  Army’s  goals  for 
the  TOPS  IOT&E  evolve  or  to  better  meet  the  infonnational  needs  of  Anny  stakeholders. 
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APPENDIX  A 


PREDICTOR  MEASURE  PSYCHOMETRIC  PROPERTIES  IN  THE 

APPLICANT  SAMPLE 


Table  A.l.  Raw  Mean  and  Standard  Deviations  for  the  TOPS  Composites  and  TAPAS  Scales 


on  the  15D-Static  and  15D-CAT-1  Forms  (June  2009-August  2011) 


1 5D-Static/CAT  - 1 

TOPS  Composite/ 

TAPAS  Scale 

(n  = 

Tier  1 
;  161,442) 

(n 

Tier  2 
=  7,095) 

Tier  1+  2 
(Combined) 

(n  =  168,537) 

M 

SD 

M 

SD 

M 

SD 

Individual  TAPAS  Scales  a 

Achievement 

.16 

.48 

.20 

.49 

.16 

.48 

Adjustment 

.00 

.57 

.07 

.58 

.00 

.57 

Adventure  Seeking 

- 

- 

- 

- 

- 

- 

Attention  Seeking 

-.21 

.53 

-.24 

.52 

-.22 

.53 

Commitment  to  Serve 

— 

— 

— 

— 

— 

— 

Cooperation 

-.06 

.37 

-.07 

.38 

-.06 

.37 

Courage 

- 

- 

- 

- 

- 

- 

Dominance 

.02 

.59 

-.02 

.60 

.02 

.59 

Even  Tempered 

.17 

.48 

.23 

.47 

.17 

.48 

Intellectual  Efficiency 

-.04 

.58 

-.01 

.57 

-.04 

.58 

Non-Delinquency 

.10 

.46 

.03 

.49 

.10 

.46 

Optimism 

.15 

.46 

.17 

.46 

.15 

.46 

Order 

-.42 

.55 

-.43 

.53 

-.42 

.55 

Physical  Conditioning 

.04 

.63 

-.10 

.60 

.04 

.62 

Responsibility 

- 

- 

- 

- 

- 

- 

Self-Control 

.07 

.53 

.13 

.53 

.07 

.53 

Selflessness 

-.20 

.43 

-.19 

.43 

-.20 

.43 

Situational  Awareness 

— 

— 

— 

— 

— 

— 

Sociability 

-.06 

.59 

-.10 

.59 

-.06 

.59 

Team  Orientation 

— 

— 

— 

— 

— 

— 

Tolerance 

-.23 

.57 

-.21 

.56 

-.23 

.57 

TOPS  Composites  (Original) 

Can-Do 

110.57 

16.51 

111.38 

16.75 

110.60 

16.53 

Will-Do 

105.54 

14.84 

104.63 

15.15 

105.50 

14.86 

TOPS  Composites  (Revised) 

Can-Do 

99.76 

20.10 

101.66 

19.21 

99.84 

20.07 

Will-Do 

100.55 

20.08 

98.06 

19.57 

100.45 

20.06 

Adaptation 

100.69 

20.04 

98.89 

19.20 

100.62 

20.01 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  and  AFQT  >  10)  with  valid 
TAPAS  score  data. 

a  Not  all  TAPAS  scales  were  administered  in  every  version;  missing  M  and  SD  indicate  that  the  scale  was  not  administered. 
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Table  A.2.  Raw  Mean  and  Standard  Deviations  for  the  TOPS  Composites  and  TAPAS  Scales 
on  Version  2,  Form  A  (August  2011-September  2012) 

15D-CAT-1A 

Tier  1+  2 


TOPS  Composite/  Tier  1  Tier  2  (Combined) 

TAPAS  Scale  (n  =  23,809)  (n  =  1,846)  (n  =  25,655) 


M 

SD 

M 

SD 

M 

SD 

Individual  TAPAS  Scales  a 

Achievement 

.23 

.49 

.29 

.48 

.24 

.49 

Adjustment 

.09 

.40 

.16 

.39 

.10 

.40 

Adventure  Seeking 

- 

- 

- 

- 

- 

- 

Attention  Seeking 

-.33 

.59 

-.29 

.61 

-.32 

.59 

Commitment  to  Serve 

— 

— 

— 

— 

— 

— 

Cooperation 

.21 

.53 

.17 

.50 

.21 

.53 

Courage 

- 

- 

- 

- 

- 

- 

Dominance 

.37 

.51 

.34 

.53 

.37 

.51 

Even  Tempered 

.27 

.49 

.36 

.50 

.28 

.49 

Intellectual  Efficiency 

.05 

.53 

.16 

.52 

.06 

.53 

Non-Delinquency 

.16 

.52 

.14 

.56 

.16 

.53 

Optimism 

.30 

.45 

.28 

.45 

.30 

.45 

Order 

-.26 

.54 

-.27 

.53 

-.26 

.54 

Physical  Conditioning 

.16 

.56 

.08 

.54 

.15 

.56 

Responsibility 

- 

- 

- 

- 

- 

- 

Self-Control 

-.25 

.48 

-.17 

.50 

-.25 

.48 

Selflessness 

.05 

.43 

.00 

.45 

.05 

.43 

Situational  Awareness 

— 

— 

— 

— 

— 

— 

Sociability 

-.18 

.56 

-.17 

.57 

-.18 

.56 

Team  Orientation 

— 

— 

— 

— 

— 

— 

Tolerance 

.00 

.52 

.02 

.51 

.00 

.52 

TOPS  Composites  (Original) 

Can-Do 

108.67 

20.10 

111.55 

20.94 

108.87 

20.18 

Will-Do 

108.96 

21.38 

109.44 

21.64 

108.99 

21.40 

TOPS  Composites  (Revised) 

Can-Do 

99.79 

19.24 

103.70 

18.98 

100.07 

19.25 

Will-Do 

99.87 

20.00 

98.43 

19.52 

99.76 

19.96 

Adaptation 

100.06 

20.58 

99.14 

19.61 

99.99 

20.51 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  and  AFQT  >10)  with  valid 
TAPAS  score  data. 

a  Not  all  TAPAS  scales  were  administered  in  every  version;  missing  M  and  SD  indicate  that  the  scale  was  not  administered. 
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Table  A. 3.  Raw  Mean  and  Standard  Deviations  for  the  TOPS  Composites  and  TAPAS  Scales 
on  Version  2,  Form  B  (August  2011-September  2012) 


15D-CAT-2B 

TOPS  Composite/ 

TAPAS  Scale 

Tier  1 

(i n  =  47,624) 

Tier  2 
(, n  =  3,768) 

Tier  1+  2 
(Combined) 

(n  =  51,392) 

M 

SD 

M 

SD 

M 

SD 

Individual  TAPAS  Scales  a 

Achievement 

.23 

.48 

21 

.48 

.23 

.48 

Adjustment 

.09 

.38 

.17 

.38 

.10 

.38 

Adventure  Seeking 

-.27 

.59 

-.23 

.57 

-.26 

.59 

Attention  Seeking 

-.32 

.58 

-.32 

.61 

-.32 

.59 

Commitment  to  Serve 

.16 

.52 

.26 

.49 

.17 

.52 

Cooperation 

.18 

.52 

.17 

.52 

.18 

.52 

Courage 

- 

- 

- 

- 

- 

- 

Dominance 

.32 

.50 

.30 

.52 

.32 

.50 

Even  Tempered 

.25 

.47 

.33 

.49 

.26 

.47 

Intellectual  Efficiency 

.03 

.52 

.10 

.52 

.03 

.52 

Non-Delinquency 

.16 

.53 

.15 

.56 

.16 

.53 

Optimism 

.25 

.44 

.24 

.44 

.25 

.44 

Order 

-.24 

.54 

-.26 

.54 

-.24 

.54 

Physical  Conditioning 

.12 

.55 

.04 

.53 

.11 

.55 

Responsibility 

- 

- 

- 

- 

— 

- 

Self-Control 

— 

— 

— 

— 

— 

— 

Selflessness 

.07 

.44 

.04 

.45 

.07 

.44 

Situational  Awareness 

.02 

.49 

.10 

.50 

.02 

.49 

Sociability 

- 

- 

- 

- 

- 

- 

Team  Orientation 

— 

— 

— 

— 

— 

— 

Tolerance 

~ 

— 

— 

— 

— 

— 

TOPS  Composites  (Original) 

Can-Do 

107.12 

19.95 

109.65 

20.70 

107.30 

20.02 

Will-Do 

107.88 

20.93 

108.74 

21.42 

107.94 

20.97 

TOPS  Composites  (Revised)  h 

Can-Do 

— 

— 

— 

— 

— 

— 

Will-Do 

99.90 

19.81 

98.39 

19.60 

99.79 

19.80 

Adaptation 

- 

- 

- 

- 

— 

— 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  and  AFQT  >10)  with  valid 
TAPAS  score  data. 

a  Not  all  TAPAS  scales  were  administered  in  every  version;  missing  M  and  SD  indicate  that  the  scale  was  not  administered. 
b  A  subset  of  the  scales  that  compose  the  New  Can-Do  and  Adaptation  composites  were  not  administered  in  this  version  of  the 
TAPAS.  Composites  could  not  be  computed. 
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Table  A.4.  Raw  Mean  and  Standard  Deviations  for  the  TOPS  Composites  and  TAPAS  Scales 
on  Version  2,  Form  C  (August  2011-September  2012) 


15D-CAT-2C 

TOPS  Composite/ 

TAPAS  Scale 

Tier  1 

(n  =  47,801) 

Tier  2 
(n  =  3,660) 

Tier  1+  2 
(Combined) 

(n  =  51,461) 

M 

SD 

M 

SD 

M 

SD 

Individual  TAPAS  Scales  a 

Achievement 

.20 

.47 

.26 

.46 

.21 

.47 

Adjustment 

.05 

.38 

.11 

.37 

.05 

.38 

Adventure  Seeking 

- 

- 

- 

- 

- 

- 

Attention  Seeking 

-.32 

.58 

-.31 

.61 

-.32 

.58 

Commitment  to  Serve 

— 

— 

— 

— 

— 

— 

Cooperation 

- 

- 

- 

- 

- 

- 

Courage 

.10 

.54 

.19 

.54 

.11 

.54 

Dominance 

.31 

.49 

.28 

.49 

.31 

.49 

Even  Tempered 

.29 

.51 

.37 

.51 

.29 

.51 

Intellectual  Efficiency 

.03 

.53 

.12 

.51 

.04 

.53 

Non-Delinquency 

.17 

.53 

.14 

.55 

.16 

.53 

Optimism 

.26 

.44 

.26 

.45 

.26 

.44 

Order 

— 

— 

— 

— 

— 

— 

Physical  Conditioning 

.11 

.55 

.01 

.51 

.10 

.55 

Responsibility 

.33 

.46 

.37 

.48 

.33 

.46 

Self-Control 

-.24 

.45 

-.18 

.45 

-.24 

.45 

Selflessness 

— 

— 

— 

— 

— 

— 

Situational  Awareness 

— 

— 

— 

— 

— 

— 

Sociability 

-.17 

.54 

-.16 

.56 

-.17 

.54 

Team  Orientation 

-.07 

.47 

-.05 

.51 

-.07 

.47 

Tolerance 

-.03 

.51 

-.01 

.51 

-.03 

.51 

TOPS  Composites  (Original) 

Can-Do 

107.72 

20.39 

110.27 

20.55 

107.90 

20.42 

Will-Do 

107.99 

21.56 

108.41 

21.83 

108.02 

21.58 

TOPS  Composites  (Revised)  h 

Can-Do 

— 

— 

— 

— 

— 

— 

Will-Do 

99.97 

19.65 

98.21 

18.98 

98.84 

19.61 

Adaptation 

— 

- 

- 

- 

- 

- 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  and  AFQT  >  10)  with  valid 
TAPAS  score  data. 

a  Not  all  TAPAS  scales  were  administered  in  every  version;  missing  M  and  SD  indicate  that  the  scale  was  not  administered. 
b  A  subset  of  the  scales  included  in  the  New  Can-Do  and  Adaptation  composites  were  not  administered  in  this  version  of  the 
TAPAS.  Composites  could  not  be  computed. 
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Table  A. 5.  Correlations  between  TOPS  Composites  and  TAPAS  Scales  with  AFQT  in  the 
TOPS  Applicant  Sample  by  Version 


15D-Static/CAT-1  15D-CAT-2  Forms 

(June  2009-August  2011) _  (August  201 1-September  2012) 


TOPS  Composite/ 

TAPAS  Scale 

Tier  1 

Tier  2 

Tier  1+  2 
(Combined) 

Tier  1 

Tier  2 

Tier  1+  2 
(Combined) 

N 

161,442 

7,095 

168,537 

23,809  - 

1,846- 

25,655- 

119,234 

9,274 

128,508 

Individual  TAPAS  Scales 

Achievement 

.10 

.04 

.10 

.04 

-.01 

.04 

Adjustment 

.11 

.10 

.11 

.13 

.12 

.13 

Adventure  Seeking 

- 

— 

~ 

.11 

.05 

■ii 

Attention  Seeking 

.11 

.08 

.11 

.01 

.00 

.01 

Commitment  to  Serve 

- 

— 

- 

-.15 

-.07 

-.15 

Cooperation 

.01 

.01 

.01 

-.13 

-.09 

-.13 

Courage 

- 

-- 

" 

.06 

.05 

.06 

Dominance 

.08 

-.01 

.08 

.12 

.03 

.12 

Even  Tempered 

.08 

.08 

.08 

.08 

.10 

.08 

Intellectual  Efficiency 

.42 

.37 

.41 

.31 

.29 

.31 

Non-Delinquency 

-.01 

.02 

-.01 

-.06 

-.02 

-.06 

Optimism 

.01 

.00 

.01 

.09 

.05 

.09 

Order 

-.18 

-.17 

-.18 

-.16 

-.16 

-.16 

Physical  Conditioning 

.05 

-.02 

.05 

.07 

-.02 

.06 

Responsibility 

-- 

-- 

-- 

.15 

.07 

.14 

Self-Control 

-.01 

.04 

-.01 

-.05 

-.02 

-.04 

Selflessness 

-.07 

-.05 

-.07 

-.08 

-.09 

-.08 

Situational  Awareness 

— 

— 

— 

.01 

.03 

.01 

Sociability 

-.09 

-.06 

-.09 

-.12 

-.09 

-.12 

Team  Orientation 

— 

— 

— 

-.11 

-.11 

-.11 

Tolerance 

-.02 

.01 

-.02 

.08 

.06 

.08 

TOPS  Composites  (Original) 

Can-Do 

.23 

.19 

.23 

.15 

.13 

.15 

Will-Do 

.05 

.02 

.05 

.04 

.02 

.04 

TOPS  Composites  (Revised) 

Can-Do 

.45 

.40 

.45 

.38 

.33 

.37 

Will-Do 

.10 

.01 

.09 

.11 

.00 

.10 

Adaptation 

.19 

.13 

.19 

.21 

.15 

.21 

Note.  Correlations  in  bold  are  statistically  significant, p  <  .01  (two-tailed).  Results  are  limited  to  the  TOPS  Applicant  Sample 


(non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with  valid  TAPAS  score  data. 
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Table  A. 6.  Descriptive  Statistics  for  AFQT,  ASVAB  Subtests,  and  Aptitude  Area  (AA)  Composites  by  Version  in  the  TOPS 
Applicant  Sample 


15D-Static/CAT-1  15D-CAT-2  Forms  A-C 

(June  2009-August  2011) _  _ (August  201 1  -September  2012) 


Composite/Subtest 

n 

M 

SD 

Min 

Max 

n 

M 

SD 

Min 

Max 

AFQT 

168,685 

57.07 

23.07 

10 

99 

128,508 

53.23 

22.05 

10 

99 

ASVAB  Subtests 

Arithmetic  Reasoning  (AR) 

167,832 

52.57 

7.69 

18 

72 

127,935 

51.23 

7.56 

23 

72 

Assembling  Objects  (AO) 

165,247 

55.20 

7.79 

25 

70 

121,118 

54.14 

7.83 

26 

70 

Auto  &  Shop  Information  (AS) 

167,831 

49.95 

9.39 

19 

86 

127,934 

48.02 

9.02 

22 

83 

Electronics  Information  (El) 

167,831 

51.99 

9.07 

16 

84 

127,934 

50.42 

8.85 

14 

84 

General  Science  (GS) 

167,832 

51.69 

8.40 

19 

76 

127,935 

50.60 

8.20 

20 

76 

Math  Knowledge  (MK) 

167,832 

53.46 

6.97 

24 

73 

127,935 

52.89 

6.70 

26 

73 

Mechanical  Comprehension  (MC) 

167,831 

53.44 

8.41 

14 

82 

127,933 

51.92 

8.22 

23 

82 

Paragraph  Comprehension  (PC) 

167,832 

52.86 

7.04 

21 

69 

127,935 

51.87 

6.84 

22 

69 

Word  Knowledge  (WK) 

167,832 

51.33 

8.04 

16 

76 

127,934 

50.04 

7.69 

16 

76 

Aptitude  Area  (AA)  Composites 

Clerical  (CL) 

167,833 

105.88 

13.90 

35 

152 

127,934 

103.26 

13.30 

56 

152 

Combat  (CO) 

167,833 

105.69 

14.84 

29 

160 

127,934 

102.60 

14.20 

54 

159 

Electronics  (EL) 

167,833 

105.47 

14.84 

29 

160 

127,934 

102.31 

14.20 

54 

159 

Field  Artillery  (FA) 

167,833 

105.85 

14.76 

28 

159 

127,934 

102.78 

14.14 

55 

159 

General  Maintenance  (GM) 

167,833 

105.17 

15.30 

28 

161 

127,934 

101.88 

14.68 

51 

160 

General  Technical  (GT) 

167,834 

104.92 

14.31 

39 

149 

127,936 

102.15 

13.77 

49 

149 

Mechanical  Maintenance  (MM) 

167,833 

104.28 

16.30 

25 

165 

127,934 

100.58 

15.64 

48 

163 

Operators  and  Food  Service  (OF) 

167,833 

105.17 

15.27 

27 

160 

127,934 

101.81 

14.65 

53 

160 

Signal  Communications  (SC) 

167,833 

105.87 

14.46 

29 

159 

127,934 

102.91 

13.82 

55 

158 

Skilled  Technical  (ST) 

167,833 

105.74 

14.48 

32 

157 

127,934 

102.78 

13.84 

55 

156 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above). 
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Table  A. 7.  Descriptive  Statistics  for  AFQT,  ASVAB  Subtests,  and  Aptitude  Area  (AA)  Composites  in  15D  Static  and  15D  CAT 
Version  1  by  Education  Tier 


Tier  1 

Tier  2 

Composite/Subtest 

n 

M 

SD 

Min 

Max 

n 

M 

SD 

Min 

Max 

AFQT 

161,575 

57.19 

23.23 

10 

99 

7,110 

54.18 

19.05 

10 

99 

ASVAB  Subtests 

Arithmetic  Reasoning  (AR) 

160,735 

52.61 

7.74 

18 

72 

7,097 

51.85 

6.35 

24 

72 

Assembling  Objects  (AO) 

158,268 

55.22 

7.80 

25 

70 

6,979 

54.79 

7.51 

26 

69 

Auto  &  Shop  Information  (AS) 

160,734 

49.84 

9.40 

19 

86 

7,097 

52.27 

8.81 

25 

81 

Electronics  Information  (El) 

160,734 

51.95 

9.11 

16 

84 

7,097 

53.04 

7.83 

18 

82 

General  Science  (GS) 

160,735 

51.7 

8.45 

19 

76 

7,097 

51.66 

6.99 

23 

74 

Math  Knowledge  (MK) 

160,735 

53.62 

6.97 

24 

73 

7,097 

49.76 

5.68 

28 

73 

Mechanical  Comprehension  (MC) 

160,734 

53.42 

8.45 

14 

82 

7,097 

53.89 

7.50 

23 

79 

Paragraph  Comprehension  (PC) 

160,735 

52.85 

7.08 

23 

69 

7,097 

53.15 

6.19 

21 

69 

Word  Knowledge  (WK) 

160,735 

51.31 

8.10 

16 

76 

7,097 

51.83 

6.67 

22 

76 

Aptitude  Area  (AA)  Composites 

Clerical  (CL) 

160,736 

105.98 

14.01 

35 

152 

7,097 

103.77 

10.86 

56 

143 

Combat  (CO) 

160,736 

105.73 

14.96 

29 

160 

7,097 

104.58 

11.88 

51 

153 

Electronics  (EL) 

160,736 

105.5 

14.96 

29 

160 

7,097 

104.85 

11.83 

52 

151 

Field  Artillery  (FA) 

160,736 

105.91 

14.88 

28 

159 

7,097 

104.66 

11.79 

51 

152 

General  Maintenance  (GM) 

160,736 

105.19 

15.42 

28 

161 

7,097 

104.87 

12.38 

48 

154 

General  Technical  (GT) 

160,737 

104.94 

14.43 

39 

149 

7,097 

104.62 

11.42 

54 

145 

Mechanical  Maintenance  (MM) 

160,736 

104.22 

16.40 

25 

165 

7,097 

105.61 

13.79 

46 

155 

Operators  and  Food  Service  (OF) 

160,736 

105.17 

15.38 

27 

160 

7,097 

105.22 

12.37 

50 

152 

Signal  Communications  (SC) 

160,736 

105.93 

14.58 

29 

159 

7,097 

104.46 

11.41 

54 

150 

Skilled  Technical  (ST) 

160,736 

105.79 

14.60 

32 

157 

7,097 

104.69 

11.42 

56 

148 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with  valid  TAP  AS  scores. 
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Table  A. 8.  Descriptive  Statistics  forAFQT,  ASVAB  Subtests,  and  Aptitude  Area  (AA)  Composites  in  TAPAS  Version  2  Forms  by 
Education  Tier 


Tier  1 

Tier  2 

Composite/Subtest 

n 

M 

SD 

Min 

Max 

n 

M 

SD 

Min 

Max 

AFQT 

119,234 

53.15 

22.28 

10 

99 

9,274 

54.21 

18.81 

10 

99 

ASVAB  Subtests 

Arithmetic  Reasoning  (AR) 

118,684 

51.19 

7.63 

23 

72 

9,251 

51.70 

6.53 

27 

72 

Assembling  Objects  (AO) 

112,261 

54.11 

7.85 

26 

70 

8,857 

54.56 

7.63 

27 

69 

Auto  &  Shop  Information  (AS) 

118,683 

47.78 

8.99 

22 

83 

9,251 

51.12 

8.81 

22 

82 

Electronics  Information  (El) 

118,683 

50.26 

8.89 

14 

84 

9,251 

52.44 

7.96 

19 

80 

General  Science  (GS) 

118,684 

50.53 

8.28 

20 

76 

9,251 

51.43 

7.10 

20 

76 

Math  Knowledge  (MK) 

118,684 

53.11 

6.73 

26 

73 

9,251 

50.07 

5.61 

28 

72 

Mechanical  Comprehension  (MC) 

118,682 

51.80 

8.25 

23 

82 

9,251 

53.40 

7.63 

24 

78 

Paragraph  Comprehension  (PC) 

118,684 

51.78 

6.89 

22 

69 

9,251 

53.08 

6.08 

27 

69 

Word  Knowledge  (WK) 

118,684 

49.92 

7.74 

16 

76 

9,251 

51.59 

6.74 

21 

76 

Aptitude  Area  (AA)  Composites 

Clerical  (CL) 

118,683 

103.24 

13.47 

56 

152 

9,251 

103.54 

10.95 

61 

148 

Combat  (CO) 

118,683 

102.50 

14.35 

54 

159 

9,251 

103.83 

12.09 

61 

155 

Electronics  (EL) 

118,683 

102.17 

14.34 

54 

159 

9,251 

104.09 

12.03 

59 

156 

Field  Artillery  (FA) 

118,683 

102.69 

14.29 

55 

159 

9,251 

103.97 

12.00 

62 

155 

General  Maintenance  (GM) 

118,683 

101.72 

14.82 

51 

160 

9,251 

103.95 

12.63 

59 

157 

General  Technical  (GT) 

118,685 

101.99 

13.92 

49 

149 

9,251 

104.27 

11.51 

60 

145 

Mechanical  Maintenance  (MM) 

118,683 

100.30 

15.73 

48 

163 

9,251 

104.22 

14.04 

56 

161 

Operators  and  Food  Service  (OF) 

118,683 

101.62 

14.78 

53 

160 

9,251 

104.28 

12.63 

59 

157 

Signal  Communications  (SC) 

118,683 

102.83 

13.97 

55 

158 

9,251 

103.91 

11.56 

60 

154 

Skilled  Technical  (ST) 

118,683 

102.68 

13.99 

55 

156 

9,251 

104.13 

11.57 

61 

152 

Note.  Results  are  limited  to  the  TOPS  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above). 
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Table  A. 9.  Correlations  among  TOPS  Composites  and  TAPAS  Scale  Scores  in  the  Applicant  Sample 


TAPAS  Scale/ 

TOPS  Composite/ 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

Individual  TAPAS  Scales 

1. 

Achievement 

2. 

Adjustment 

.10 

3. 

Adventure  Seeking 

.10 

.15 

4. 

Attention  Seeking 

.04 

.09 

.17 

5. 

Commitment  to  Serve 

.11 

.05 

.04 

-.01 

6. 

Cooperation 

.10 

.10 

-.13 

-.05 

.04 

7. 

Courage 

.21 

.15 

-- 

.10 

- 

-- 

8. 

Dominance 

.31 

.11 

.13 

.19 

.05 

.03 

.21 

9. 

Even  Tempered 

.11 

.21 

-.04 

-.04 

.05 

.27 

.04 

-.02 

10. 

Intellectual  Efficiency 

.25 

.18 

.07 

.08 

.01 

.01 

.18 

.26 

.09 

11. 

Non-Delinquency 

.19 

.01 

-.17 

-.14 

.12 

.22 

.03 

.00 

.22 

.03 

12. 

Optimism 

.17 

.27 

.02 

.10 

.01 

.16 

.03 

.16 

.19 

.10 

.11 

13. 

Order 

.17 

-.06 

-.09 

-.08 

.05 

.08 

- 

.09 

.00 

.04 

.12 

.01 

14. 

Physical  Conditioning 

.17 

.06 

.25 

.10 

.00 

-.02 

.12 

.19 

-.08 

.05 

-.04 

.07 

.04 

15. 

Responsibility 

.30 

.12 

- 

-.05 

- 

-- 

.14 

.15 

.15 

.16 

.23 

.15 

- 

.04 

16. 

Self-Control 

.20 

.05 

-- 

-.08 

" 

.08 

.10 

-.03 

.17 

.14 

.22 

.04 

.16 

-.06 

.22 

17. 

Selflessness 

.12 

.00 

-.03 

-.09 

.04 

.25 

- 

.09 

.13 

.00 

.16 

.08 

.09 

-.02 

— 

18. 

Situational  Awareness 

.19 

.14 

.10 

.04 

.07 

.00 

- 

.11 

.10 

.25 

.11 

.08 

.15 

.05 

— 

19. 

Sociability 

.05 

.09 

-- 

.34 

-- 

.13 

.08 

.19 

.01 

.02 

-.04 

.17 

-.04 

.10 

.03 

20. 

Team  Orientation 

.06 

.04 

- 

.12 

— 

- 

.03 

.10 

.06 

-.05 

.05 

.05 

- 

.05 

.02 

21. 

Tolerance 

.10 

.03 

- 

.02 

— 

.16 

.03 

.09 

.14 

.09 

.08 

.11 

.04 

-.04 

.08 

22. 

Can-Do  (current) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

23. 

Will-Do  (current) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

24. 

Can-Do  (revised) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

25. 

Will-Do  (revised) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

26. 

Adaptation  (new) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

Note.  Results  are  limited  to  the  Applicant  Sample  (Non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with  valid  TAPAS  15D.  15D-CAT,  v5,  v7  or  v8 
score  data,  n  =  29,745  -  245,653.  Correlations  in  bold  are  statistically  significant,  p  <  .01  (two-tailed). 
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Table  A. 9.  (Continued)) 

TAPAS  Scale/ 

TOPS  Composite/ _ 16  17  18  19  20  21  22  23  24  25 

Individual  TAPAS  Scales 

1.  Achievement 

2.  Adjustment 

3.  Adventure  Seeking 

4.  Attention  Seeking 

5.  Commitment  to  Serve 

6.  Cooperation 

7.  Courage 

8.  Dominance 

9.  Even  Tempered 

10.  Intellectual  Efficiency 

11.  Non-Delinquency 

12.  Optimism 

13.  Order 

14.  Physical  Conditioning 

15.  Responsibility 

16.  Self-Control 

17.  Selflessness  .05 


18. 

Situational  Awareness 

” 

.04 

19. 

Sociability 

-.06 

.06 

-- 

20. 

Team  Orientation 

.05 

- 

- 

.22 

21. 

Tolerance 

.06 

.32 

- 

.10 

.08 

22. 

Can-Do  (current) 

— 

— 

— 

— 

— 

— 

23. 

Will-Do  (current) 

— 

— 

— 

— 

— 

— 

.71 

24. 

Can-Do  (revised) 

— 

— 

— 

— 

— 

— 

.50 

.22 

25. 

Will-Do  (revised) 

— 

— 

— 

— 

— 

— 

.47 

.51 

.16 

26. 

Adaptation  (new) 

— 

— 

— 

— 

— 

— 

.21 

.47 

.40 

.48 

Note.  Results  are  limited  to  the  Applicant  Sample  (Non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above) 
with  valid  TAPAS  15D.  15D-CAT,  v5,  v7  or  v8  score  data,  n  =  29,745  -  245,653.  Correlations  in  bold  are  statistically 
significant,  p  <  .01  (two-tailed). 
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Table  A.  10.  Correlations  among  TOPS  Composites  and  TAPAS  Scale  Scores  in  the  Applicant  Sample  by  Education  Tier 


TAPAS  Scale/ 

TOPS  Composite/ 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

Individual  TAPAS  Scales 

1 .  Achievement 

.10 

.10 

.04 

.11 

.10 

.21 

.31 

.11 

.25 

.19 

.17 

.17 

.17 

.30 

2. 

Adjustment 

.09 

.15 

.09 

.04 

.10 

.15 

.12 

.21 

.18 

.01 

.27 

-.06 

.07 

.12 

3. 

Adventure  Seeking 

.09 

.12 

.17 

.03 

-.13 

- 

.13 

-.04 

.07 

-.17 

.02 

-.09 

.25 

-- 

4. 

Attention  Seeking 

.05 

.10 

.19 

-.01 

-.05 

.10 

.18 

-.04 

.08 

-.15 

.10 

-.08 

.10 

-.05 

5. 

Commitment  to  Serve 

.14 

.04 

.11 

.03 

.04 

- 

.05 

.05 

.01 

.12 

.01 

.05 

.00 

— 

6. 

Cooperation 

.11 

.11 

-.14 

-.04 

.03 

-- 

.03 

.27 

.01 

.22 

.16 

.08 

-.02 

- 

7. 

Courage 

.21 

.12 

- 

.12 

- 

- 

.21 

.04 

.18 

.03 

.03 

-- 

.12 

.14 

8. 

Dominance 

.28 

.10 

.16 

.19 

.06 

.03 

.22 

-.02 

.26 

.00 

.17 

.09 

.19 

.15 

9. 

Even  Tempered 

.10 

.19 

-.08 

-.05 

.05 

.29 

.02 

-.03 

.09 

.22 

.19 

.00 

-.07 

.15 

10. 

Intellectual  Efficiency 

.23 

.16 

.05 

.07 

.04 

.03 

.18 

.25 

.09 

.03 

.10 

.04 

.06 

.16 

11. 

Non-Delinquency 

.24 

.05 

-.17 

-.12 

.11 

.26 

.08 

.02 

.27 

.07 

.11 

.12 

-.04 

.22 

12. 

Optimism 

.15 

.26 

.04 

.09 

.03 

.17 

.04 

.12 

.21 

.09 

.16 

.01 

.08 

.15 

13. 

Order 

.17 

-.04 

-.05 

-.06 

.05 

.08 

- 

.10 

.02 

.06 

.13 

-.02 

.04 

— 

14. 

Physical  Conditioning 

.17 

.04 

.24 

.09 

.06 

.00 

.12 

.19 

-.07 

.07 

-.03 

.05 

.10 

.05 

15. 

Responsibility 

.30 

.10 

-- 

-.05 

- 

- 

.19 

.17 

.16 

.16 

.30 

.14 

- 

.02 

16. 

Self-Control 

.20 

.08 

- 

-.07 

— 

.08 

.10 

-.03 

.16 

.13 

.24 

.05 

.15 

-.05 

.24 

17. 

Selflessness 

.13 

.00 

-.02 

-.07 

.04 

.26 

- 

.06 

.14 

.02 

.21 

.10 

.10 

.01 

— 

18. 

Situational  Awareness 

.22 

.14 

.11 

.03 

.10 

-.02 

- 

.14 

.09 

.24 

.11 

.08 

.14 

.07 

— 

19. 

Sociability 

.05 

.10 

-- 

.35 

-- 

.14 

.09 

.18 

.04 

.01 

-.03 

.17 

-.06 

.07 

.05 

20. 

Team  Orientation 

.07 

.05 

- 

.13 

— 

- 

.03 

.12 

.07 

-.03 

.10 

.04 

- 

.01 

.06 

21. 

Tolerance 

.09 

.05 

- 

.04 

— 

.16 

.02 

.07 

.14 

.10 

.13 

.11 

.03 

-.03 

.08 

22. 

Can-Do  (current) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

23. 

Will-Do  (current) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

24. 

Can-Do  (revised) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

25. 

Will-Do  (revised) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

26. 

Adaptation  (new) 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

Note.  Results  are  limited  to  the  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with  valid  TAPAS  score  data.  Correlations  above  the 
diagonal  are  for  Education  Tier  1  applicants,  n  =  47,624  -  280,676.  Correlations  below  the  diagonal  are  for  Education  Tier  2  applicants,  n  =  3,660  -  16,369.  Correlations  in  bold 
are  statistically  significant,/)  <  .01  (two-tailed). 
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Table  A.  10.  (Continued) 


TAPAS  Scale/ 

TOPS  Composite/ 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

Individual  TAPAS  Scales 

1 .  Achievement 

.20 

.12 

.19 

.05 

.06 

.11 

.61 

.56 

.27 

.65 

.07 

2. 

Adjustment 

.05 

.00 

.14 

.09 

.04 

.03 

.25 

.10 

.17 

.16 

.13 

3. 

Adventure  Seeking 

-- 

-.03 

.10 

-- 

- 

-- 

-.02 

-.04 

-- 

.24 

-- 

4. 

Attention  Seeking 

-.08 

-.09 

.04 

.34 

.12 

.02 

.03 

-.42 

-.04 

.17 

-.01 

5. 

Commitment  to  Serve 

- 

.04 

.07 

- 

-- 

- 

.11 

.12 

-- 

.07 

- 

6. 

Cooperation 

.08 

.25 

.00 

.13 

- 

.16 

.22 

.24 

-.03 

.02 

.02 

7. 

Courage 

.10 

- 

-- 

.08 

.03 

.03 

.17 

.11 

-- 

.24 

- 

8. 

Dominance 

-.03 

.09 

.11 

.19 

.10 

.09 

.19 

.09 

.12 

.52 

.00 

9. 

Even  Tempered 

.17 

.13 

.10 

.01 

.06 

.14 

.53 

.49 

.20 

.01 

.36 

10. 

Intellectual  Efficiency 

.14 

.00 

.25 

.02 

-.05 

.09 

.51 

.13 

.79 

.23 

.06 

11. 

Non-Delinquency 

.22 

.16 

.12 

-.04 

.04 

.07 

.55 

.61 

.03 

.06 

.03 

12. 

Optimism 

.04 

.08 

.09 

.17 

.06 

.10 

.51 

.17 

.04 

.35 

.07 

13. 

Order 

.16 

.09 

.15 

-.04 

— 

.04 

.10 

.16 

-.34 

.10 

-.43 

14. 

Physical  Conditioning 

-.06 

-.02 

.05 

.10 

.06 

-.04 

.05 

.35 

.01 

.76 

.68 

15. 

Responsibility 

.22 

-- 

-- 

.03 

.02 

.08 

.35 

.30 

-- 

.23 

-- 

16. 

Self-Control 

.05 

-- 

-.06 

.05 

.06 

.34 

.26 

.14 

.10 

.01 

17. 

Selflessness 

.04 

.04 

.06 

— 

.32 

.12 

.17 

-.09 

.03 

-.03 

18. 

Situational  Awareness 

- 

.04 

- 

-- 

- 

.26 

.16 

- 

.16 

-- 

19. 

Sociability 

-.06 

.06 

- 

.22 

.10 

.09 

-.08 

-.37 

.19 

-.26 

20. 

Team  Orientation 

.06 

- 

-- 

.24 

.07 

.06 

.04 

- 

.11 

- 

21. 

Tolerance 

.04 

.30 

- 

.11 

.10 

.15 

.09 

-.17 

.04 

-.05 

22. 

Can-Do  (current) 

— 

— 

— 

— 

-- 

— 

.71 

.50 

.47 

.21 

23. 

Will-Do  (current) 

— 

— 

— 

— 

— 

.73 

.22 

.52 

.47 

24. 

Can-Do  (revised) 

— 

— 

— 

— 

-- 

— 

.46 

.23 

.16 

.40 

25. 

Will-Do  (revised) 

— 

— 

— 

— 

-- 

— 

.46 

.49 

.14 

.48 

26. 

Adaptation  (new) 

— 

— 

— 

— 

— 

— 

.20 

.45 

.39 

.47 

Note.  Results  are  limited  to  the  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with 
valid  TAP  AS  score  data.  Correlations  above  the  diagonal  are  for  Education  Tier  1  applicants,  n  =  47,624  -  280,676.  Correlations 
below  the  diagonal  are  for  Education  Tier  2  applicants,  n  =  3,660  -  16,369.  Correlations  in  bold  are  statistically  significant,/;  <  .01 
(two-tailed).  For  test  security  reasons,  correlations  between  the  TAP  AS  scales  and  TOPS  composites  are  excluded. 
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Table  A.l  1.  Correlations  among  AFQT,  ASVAB  Subtests,  and  Aptitude  Area  (AA)  Composite  Scores  in  the  TOPS  Applicant 
Sample 


Composite/Subtest 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

1. 

AFQT 

ASVAB  Subtests 

2. 

Arithmetic  Reasoning  (AR) 

.82 

3. 

Assembling  Objects  (AO) 

.44 

.47 

4. 

Auto  &  Shop  Info  (AS) 

.37 

.32 

.27 

5. 

Electronics  Information  (El) 

.59 

.47 

.36 

.68 

6. 

General  Science  (GS) 

.73 

.54 

.36 

.51 

.68 

7. 

Math  Knowledge  (MK) 

.71 

.68 

.38 

.07 

.28 

.42 

8. 

Mech  Comprehension  (MC) 

.65 

.60 

.52 

.62 

.68 

.66 

.40 

9. 

Para  Comprehension  (PC) 

.80 

.55 

.35 

.35 

.53 

.64 

.41 

.54 

10. 

Word  Knowledge  (WK) 

Aptitude  Area  (AA)  Composites 

.81 

.48 

.28 

.41 

.59 

.72 

.34 

.55 

.69 

11. 

Clerical  (CL) 

.96 

.90 

.50 

.43 

.64 

.73 

.77 

.72 

.75 

.74 

12. 

Combat  (CO) 

.88 

.79 

.51 

.68 

.79 

.81 

.66 

.85 

.70 

.71 

.94 

13. 

Electronics  (EL) 

.90 

.80 

.50 

.67 

.81 

.80 

.63 

.83 

.73 

.75 

.95 

1.0 

14. 

Field  Artillery  (FA) 

.89 

.82 

.52 

.65 

.77 

.79 

.68 

.86 

.70 

.71 

.95 

1.0 

.99 

15. 

General  Maintenance  (GM) 

.85 

.78 

.50 

.73 

.82 

.81 

.60 

.85 

.68 

.70 

.91 

.99 

.99 

.99 

16. 

General  Technical  (GT) 

.96 

.88 

.46 

.42 

.62 

.73 

.62 

.68 

.80 

.81 

.97 

.88 

.91 

.90 

.87 

17. 

Mech  Maintenance  (MM) 

.74 

.67 

.47 

.86 

.84 

.75 

.44 

.85 

.62 

.65 

.81 

.95 

.95 

.94 

.97 

.77 

18. 

Operators  &  Food  (OF) 

.86 

.79 

.50 

.73 

.80 

.80 

.58 

.86 

.70 

.72 

.92 

.99 

.99 

.99 

1.0 

.89 

.97 

19. 

Signal  Communications  (SC) 

.92 

.82 

.51 

.60 

.78 

.78 

.70 

.81 

.73 

.75 

.97 

.99 

.99 

.99 

.98 

.93 

.92 

.98 

20. 

Skilled  Technical  (ST) 

.94 

.83 

.51 

.59 

.75 

.81 

.68 

.82 

.76 

.78 

.97 

.99 

.99 

.99 

.97 

.95 

.91 

.98 

1.0 

Note.  Results  are  limited  to  the  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with  valid  TAP  AS  score  data,  n  =  286,365-297,193. 
All  correlations  are  statistically  significant,/)  <  .01  (two-tailed). 
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Table  A.  12.  Correlations  among  AFQT,  ASVAB  Subtests,  and  Aptitude  Area  (AA)  Composite  Scores  by  Education  Tier 


Composite/Subtest 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

1. 

AFQT 

ASVAB  Subtests 

.82 

.45 

.37 

.59 

.72 

.72 

.64 

.79 

.80 

.97 

.88 

.90 

.89 

.85 

.96 

.75 

.87 

.93 

.94 

2. 

Arithmetic  Reasoning  (AR) 

.77 

.47 

.31 

.46 

.53 

.68 

.58 

.53 

.47 

.90 

.79 

.80 

.81 

.78 

.88 

.67 

.79 

.82 

.83 

3. 

Assembling  Objects  (AO) 

.40 

.42 

.27 

.35 

.36 

.38 

.51 

.34 

.28 

.50 

.51 

.50 

.52 

.50 

.46 

.46 

.50 

.51 

.51 

4. 

Auto  &  Shop  Info  (AS) 

.31 

.25 

.23 

.66 

.50 

.08 

.61 

.34 

.41 

.43 

.67 

.67 

.65 

.73 

.42 

.86 

.72 

.60 

.59 

5. 

Electronics  Information  (El) 

.53 

.37 

.31 

.64 

.67 

.28 

.67 

.52 

.58 

.63 

.78 

.80 

.76 

.81 

.61 

.83 

.79 

.77 

.74 

6. 

General  Science  (GS) 

.67 

.43 

.30 

.46 

.64 

.41 

.65 

.62 

.71 

.72 

.80 

.79 

.79 

.80 

.72 

.74 

.79 

.77 

.80 

7. 

Math  Knowledge  (MK) 

.65 

.62 

.32 

.00 

.19 

.31 

.40 

.41 

.33 

.77 

.67 

.64 

.68 

.60 

.62 

.45 

.59 

.70 

.68 

8. 

Mech  Comprehension  (MC) 

.58 

.51 

.47 

.59 

.62 

.60 

.30 

.53 

.54 

.71 

.85 

.82 

.85 

.84 

.67 

.85 

.85 

.80 

.81 

9. 

Para  Comprehension  (PC) 

.75 

.45 

.29 

.28 

.44 

.54 

.30 

.45 

.68 

.74 

.68 

.72 

.69 

.67 

.79 

.61 

.69 

.72 

.75 

10. 

Word  Knowledge  (WK) 
Aptitude  Area  (AA)  Composites 

.76 

.35 

.23 

.35 

.54 

.66 

.20 

.48 

.60 

.73 

.70 

.74 

.70 

.69 

.81 

.65 

.71 

.74 

.77 

11. 

Clerical  (CL) 

.95 

.87 

.46 

.39 

.58 

.66 

.71 

.67 

.68 

.67 

.94 

.95 

.95 

.91 

.97 

.81 

.92 

.97 

.97 

12. 

Combat  (CO) 

.84 

.72 

.47 

.68 

.76 

.76 

.58 

.83 

.61 

.64 

.91 

.99 

1.0 

.99 

.88 

.95 

.99 

.99 

.99 

13. 

Electronics  (EL) 

.87 

.74 

.46 

.67 

.78 

.75 

.55 

.80 

.65 

.68 

.93 

.99 

.99 

.99 

.91 

.95 

.99 

.99 

.99 

14. 

Field  Artillery  (FA) 

.85 

.76 

.48 

.65 

.73 

.74 

.60 

.84 

.61 

.63 

.93 

1.0 

.99 

.99 

.90 

.94 

.99 

.99 

.99 

15. 

General  Maintenance  (GM) 

.80 

.71 

.46 

.74 

.80 

.76 

.51 

.82 

.59 

.62 

.88 

.99 

.99 

.99 

.87 

.97 

1.0 

.98 

.97 

16. 

General  Technical  (GT) 

.95 

.86 

.41 

.36 

.55 

.66 

.53 

.61 

.74 

.76 

.96 

.84 

.88 

.86 

.82 

.77 

.89 

.92 

.94 

17. 

Mech  Maintenance  (MM) 

.68 

.59 

.42 

.87 

.81 

.68 

.34 

.83 

.52 

.57 

.76 

.95 

.94 

.93 

.97 

.71 

.97 

.92 

.91 

18. 

Operators  &  Food  (OF) 

.82 

.73 

.46 

.73 

.77 

.74 

.49 

.84 

.61 

.65 

.90 

.99 

.99 

.99 

1.0 

.85 

.97 

.98 

.98 

19. 

Signal  Communications  (SC) 

.90 

.77 

.47 

.59 

.75 

.72 

.63 

.78 

.66 

.68 

.96 

.99 

.99 

.99 

.97 

.90 

.90 

.97 

1.0 

20. 

Skilled  Technical  (ST) 

.92 

.78 

.47 

.57 

.72 

.76 

.59 

.79 

.69 

.72 

.96 

.98 

.99 

.99 

.97 

.92 

.89 

.97 

.99 

Note.  Results  are  limited  to  the  Applicant  Sample  (non-prior  service.  Education  Tier  1  and  2,  AFQT  Category  IV  and  above)  with  valid  TAP  AS  score  data.  Correlations  above  the  diagonal 
are  for  Education  Tier  1  applicants,  n  =  1 12,261  -  1 19,234.  Correlations  below  the  diagonal  are  for  Education  Tier  2  applicants,  n  =  8,857  -  9,274.  All  correlations  are  statistically 
significant,  p<  .01  (two-tailed). 
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CORRELATIONS  AMONG  CRITERION  MEASURES  IN  THE  IMT  AND  IN-UNIT 

VALIDATION  SAMPLES 


Table  B.  1 .  Correlations  among  the  Performance  Rating  Scales  (PRS)  in  the  IMT  Validation 
Sample 


Domain/PRS 

1 

2 

3 

4 

5 

6 

Army-  Wide 

1. 

Adjustment  to  the  Army 

2. 

Effort  &  Personal  Discipline 

.75 

3. 

MOS  Qualification  Knowledge 

.72 

.67 

4. 

Physical  Fitness  &  Bearing 

.67 

.70 

.59 

5. 

Working  with  Others 

.73 

.76 

.68 

.64 

6. 

Overall  Performance 

.59 

.60 

.56 

.54 

.56 

MOS-Speciftc 

7. 

1 1B/C/X  +  18X 

.66 

.63 

.68 

.60 

.65 

.55 

8. 

19K 

.66 

.65 

.79 

.63 

.37 

.63 

9. 

31B 

.68 

.64 

.72 

.53 

.68 

.56 

10. 

68W 

.70 

.63 

.71 

.56 

.66 

.39 

11. 

88M 

.67 

.63 

.64 

.65 

.64 

.61 

12. 

All  MOS  Combined 

.68 

.63 

.70 

.57 

.65 

.51 

Note.  Army-wide  PRS:  n  =  4,991-5,342.  MOS-specific  PRS:  1  IB,  n  =  1,416-1,419;  19K,  n  =  83;  31B,  n  =  1,009-1,014;  68W,  n  = 
635-657;  88M,  n  =  108-109;  All  MOS  Combined,  n  =  3,299-3,324.  Ratings  on  PRS  range  from  1  and  5.  PRS  ratings  from 
supervisors  with  a  familiarity  rating  of  1  (“I  have  had  little  opportunity  to  observe  this  Soldier”)  were  excluded  from  analyses. 

All  correlations  are  statistically  significant  (p  <  .05,  one-tailed). 


Table  B.2.  Correlations  among  the  Performance  Rating  Scales  ( PRS)  in  the  In-Unit 
Validation  Sample 


Domain/PRS 

1 

2 

3 

4 

5 

1. 

Can  Do  3 

2. 

Effort  &  Personal  Discipline  a 

.79 

3. 

Physical  Fitness  &  Bearing 

.54 

.56 

4. 

Self-Management J 

.76 

.75 

.60 

5. 

Working  with  Others  a 

.79 

.79 

.53 

.75 

6. 

Overall  Leadership  Potential 

.67 

.67 

.61 

.67 

.63 

Note.  Army-wide  PRS,  n  =  708-734.  Ratings  on  PRS  range  from  1  and  7.  PRS  ratings  from  supervisors  with  a  familiarity  rating 
of  1  (“I  have  had  little  opportunity  to  observe  this  Soldier”)  were  excluded  from  analyses.  All  correlations  are  statistically 
significant  (p  <  .05,  one-tailed). 

a  Ratings  composite  comprises  two  or  more  Army-wide  PRS. 
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Table  B.3.  Correlations  among  the  Army  Life  Questionnaire  (ALQ)  Scales  in  the  IMT  and  In-Unit  Validation  Samples 


Domain/Scale 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

1. 

Affective  Commitment 

.79 

.40 

.49 

— 

.61 

.52 

-.56 

— 

-.14 

.05 

— 

2. 

Army  Fit 

.84 

.42 

.50 

- 

.62 

.55 

-.64 

- 

-.14 

.04 

- 

3. 

MOS  Fit 

.47 

.48 

.54 

- 

.27 

.18 

-.31 

- 

-.12 

.01 

- 

4. 

MOS  Satisfaction 

— 

— 

— 

— 

.37 

.29 

-.36 

— 

-.10 

.02 

— 

5. 

Normative  Commitment 

.69 

.73 

.41 

— 

— 

- 

— 

— 

— 

— 

— 

6. 

Army  Career  Intentions 

.56 

.52 

.25 

— 

.42 

.78 

-.50 

— 

-.11 

.06 

— 

7. 

Army  Reenlistment  Intentions 

.53 

.53 

.28 

- 

.46 

.85 

-.45 

- 

-.09 

.07 

- 

8. 

Attrition  Cognition 

-.63 

-.69 

-.41 

- 

-.74 

-.46 

-.49 

- 

.24 

-.09 

- 

9. 

Army  Life  Adjustment 

.45 

.61 

.34 

- 

.46 

.36 

.39 

-.53 

- 

- 

- 

10. 

Disciplinary  Incidents  (#) 

-.07 

-.09 

-.07 

- 

-.06 

-.04 

-.05 

.10 

-.16 

-.04 

- 

11. 

Last  APFT  Score 

.03 

.09 

.07 

- 

.07 

.04 

.04 

-.11 

.22 

-.13 

- 

12. 

Training  Achievement 

.05 

.06 

.05 

— 

.00 

.08 

.06 

-.03 

.12 

-.07 

.23 

13. 

Training  Restart  (#) 

-.05 

-.08 

-.08 

- 

-.06 

-.04 

-.04 

.10 

-.19 

.19 

-.26 

-.12 

Note.  Correlations  below  the  diagonal  reflect  the  IMT  ALQ,  n  =  15,037-16,835.  Correlations  above  the  diagonal  reflect  the  in-unit  ALQ,  n  =  1,008-1,034.  Correlations  in  bold  are 
statistically  significant,/)  <  .01  (two-tailed). 
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Table  B.4.  Correlations  between  the  Army  Life  Questionnaire  (ALQ)  Scales  and  Job  Knowledge  Tests  (JKTs)  in  the  IMT  and  In- 
Unit  Validation  Samples 


IMT/ln-Unit  JKTs 

Setting/ ALQ  Scale 

WTBD 

All  MOS 

Combined 

1  IB 

19K 

31B 

42A 

68W 

88M 

91B 

IMT 

Affective  Commitment 

.09 

.03 

.10 

.18 

.06 

.11 

.03 

.04 

.21 

Army  Fit 

.15 

.07 

.16 

.21 

.07 

.07 

.10 

.07 

.21 

MOS  Fit 

.13 

.12 

.12 

.18 

.04 

-.04 

.16 

.00 

.32 

Normative  Commitment 

.22 

.16 

.22 

.21 

.14 

.10 

.17 

.16 

.26 

Army  Career  Intentions 

-.03 

-.03 

.01 

.04 

.01 

.02 

-.02 

-.03 

.04 

Army  Reenlistment  Intentions 

.03 

.02 

.05 

.09 

.04 

.09 

.02 

.03 

.08 

Attrition  Cognitions 

-.18 

-.12 

-.18 

-.17 

-.11 

-.10 

-.15 

-.11 

-.18 

Army  Life  Adjustment 

.13 

.08 

.13 

.10 

.12 

.04 

.12 

.09 

.14 

Disciplinary  Incidents  (#) 

-.03 

.01 

-.01 

.04 

-.08 

-.04 

-.04 

.00 

-.03 

Last  APFT  Score 

.07 

.05 

.05 

.01 

.00 

-.05 

.01 

-.01 

-.01 

Training  Achievement 

-.10 

-.11 

-.13 

-.04 

-.06 

-.09 

.01 

-.13 

-.20 

Training  Restart  (#) 

-.07 

-.02 

-.07 

-.07 

-.10 

-.09 

-.06 

-.08 

.00 

In-Unit 

Affective  Commitment 

.07 

-.01 

.10 

— 

— 

— 

— 

-.08 

— 

Army  Fit 

.07 

.06 

.11 

- 

- 

- 

- 

.05 

- 

MOS  Fit 

.10 

.02 

.08 

- 

- 

- 

- 

-.18 

- 

MOS  Satisfaction 

-.07 

-.05 

.04 

- 

- 

- 

- 

-.12 

- 

Army  Career  Intentions 

.02 

-.06 

.02 

- 

- 

- 

- 

.02 

- 

Army  Reenlistment  Intentions 

.03 

-.01 

.06 

- 

- 

- 

- 

-.08 

- 

Attrition  Cognitions 

-.11 

-.10 

-.13 

- 

- 

- 

- 

-.29 

- 

Disciplinary  Incidents  (#) 

-.05 

-.16 

-.22 

- 

- 

- 

- 

-.19 

- 

Last  APFT  Score 

.01 

-.08 

-.10 

- 

- 

- 

- 

-.01 

- 

Note.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  IMT:  All  MOS  Combined,  n  =  12,095-13,409;  1  IB,  n  =  5,289-5,325;  19K,  n  =  293-301;  31B,  n  =  2,134-2,525;  42A,  n  =  243-244; 
68W,  n  =  2,183-2,686;  88M,  n  =  1,749-2,069;  91B,  n  =  159-259;  WTBD,  n  =  14,346-15,947.  In-Unit:  All  MOS  Combined,  n  =  361-367;  1  IB,  n  =  156-159;  88M,  n  =  50-52; 
WTBD,  n  =  985-1,008.  Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 
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Table  B.5.  Correlations  between  the  Army  Life  Questionnaire  (ALQ)  Scales  and  Performance  Rating  Scales  ( PRS )  in  the  IMT 
Validation  Sample 


AFF 

Army 

MOS 

NORM 

CAR 

RENL 

ATT 

AL 

DIS 

Last 

TRN 

TRN 

Domain/PRS 

COM 

Fit 

Fit 

COM 

INT 

INT 

COG 

ADJ 

INC 

APFT 

ACH 

RES 

Army-  Wide 

Adjustment  to  the  Army 

.05 

.07 

.07 

.03 

.02 

.03 

-.06 

.07 

-.11 

.15 

.10 

-.05 

Effort  &  Personal  Discipline 

.05 

.07 

.07 

.04 

.01 

.03 

-.06 

.07 

-.10 

.15 

.08 

-.03 

MOS  Qualification  Knowledge 

.04 

.05 

.06 

.03 

.01 

.02 

-.03 

.07 

-.08 

.12 

.07 

-.04 

Physical  Fitness  &  Bearing 

.04 

.06 

.06 

.03 

.03 

.04 

-.06 

.10 

-.09 

.29 

.14 

-.10 

Working  with  Others 

.03 

.04 

.05 

.03 

.02 

.03 

-.04 

.05 

-.06 

.11 

.07 

-.02 

Overall  Performance 

.05 

.07 

.06 

.05 

.02 

.03 

-.08 

.12 

-.13 

.21 

.15 

-.10 

MOS-Specific 

All  MOS  Combined 

.07 

.09 

.08 

.07 

.04 

.05 

-.09 

.07 

-.06 

.10 

.06 

-.02 

11B 

.04 

.05 

.10 

.06 

.02 

.03 

-.07 

.03 

-.04 

.10 

.05 

-.01 

19K 

.10 

.12 

.16 

.10 

.21 

.18 

-.17 

.24 

-.28 

.12 

-.07 

-.30 

25B 

.02 

.06 

.19 

.04 

-.09 

-.05 

.02 

.14 

-.03 

-.02 

-.01 

-.03 

31B 

.10 

.12 

.06 

.07 

.07 

.06 

-.10 

.13 

-.14 

.07 

.14 

-.02 

68W 

.07 

.07 

.04 

.08 

.07 

.08 

-.09 

.09 

-.01 

.07 

-.01 

.02 

88M 

.00 

.02 

-.06 

-.05 

-.08 

-.08 

.14 

-.12 

-.07 

.19 

-.08 

.02 

Note.  AFF  COM  =  Affective  Commitment;  NORM  COM  =  Nonnative  Commitment;  CAR  INT  =  Army  Career  Intentions;  RENL  INT  =  Army  Reenlistment  Intentions;  ATT 
COG  =  Attrition  Cognitions;  AL  ADJ  =  Army  Life  Adjustment;  DIS  INC  =  Disciplinary  Incidents  (#  of);  Last  APFT  =  Last  APFT  Score;  TRN  ACH  =  Training  Achievements  (# 
of);  TRN  RES  =  Training  Restart  (#  of).  Army-wide  PRS:  n  =  4,510-5,222.  MOS-specific  PRS:  All  MOS  Combined,  n  =  3,161-3,520;  1  IB,  n  =  1,446-1,457;  19K,  n  =  81-83;  25B, 
n  =  183-185;  3  IB,  n=  929-1,053;  68W,  n  =  584-772;  88M,  n  =  88-1 14.  Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 
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Table  B.6.  Correlations  between  the  Army  Life  Questionnaire  (ALQ)  Scales  and  Performance  Rating  Scales  ( PRS )  in  the  In-Unit 
Validation  Sample 


In-Unit  ALQ  Scale 


AFF 

Army 

MOS 

MOS 

CAR 

RENL 

ATT 

DIS 

Last 

Domain/PRS 

COM 

Fit 

Fit 

SAT 

INT 

INT 

COG 

INC 

APFT 

Army-  Wide 

Can  Do  a 

.06 

.04 

.01 

-.07 

.04 

.08 

-.08 

-.20 

.11 

Effort  &  Personal  Discipline  a 

.07 

.06 

.04 

-.05 

.02 

.03 

-.08 

-.27 

.12 

Physical  Fitness  &  Bearing 

.03 

.05 

.00 

-.02 

.03 

.03 

-.10 

-.20 

.36 

Self-Management a 

.09 

.07 

.03 

-.03 

.07 

.06 

-.13 

-.30 

.12 

Working  with  Others  a 

.04 

.03 

.04 

-.04 

.03 

.04 

-.08 

-.23 

.10 

Overall  Leadership  Potential 

.11 

.11 

.03 

.01 

.06 

.06 

-.14 

-.26 

.19 

Note.  AFFCOM  =  Affective  Commitment;  MOS  SAT  =  MOS  Satisfaction;  CAR  INT  =  Army  Career  Intentions;  RENL  INT  =  Army  Reenlistment  Intentions;  ATT  COG  = 
Attrition  Cognitions;  DIS  INC  =  Disciplinary  Incidents  (#  of);  Last  APFT  =  Last  APFT  Score.  Army-wide  PRS,  n  =  686-725.  Correlations  in  bold  are  statistically  significant  (p  < 
.05,  two-tailed). 

“Ratings  composite  comprises  two  or  more  Army-wide  PRS. 


Table  B.7.  Correlations  between  the  Job  Knowledge  Tests  (JKTs)  and  Performance  Rating 
Scales  (PRS)  in  the  IMT  Validation  Sample 


Domain/PRS 

IMT  JKT 

All  MOS 

Combined 

1  IB 

19K 

31B 

42A 

68W 

88M 

WTBD 

Army-  Wide 

Adjustment  to  the  Army 

.01 

.05 

.31 

.04 

.26 

-.05 

.27 

.06 

Effort  &  Personal  Discipline 

.05 

.05 

.27 

.04 

.29 

-.01 

.26 

.08 

MOS  Qualification  Knowledge 

.04 

.07 

.31 

.04 

.37 

-.02 

.10 

.07 

Physical  Fitness  &  Bearing 

.03 

.07 

.27 

.03 

.22 

-.03 

.22 

.06 

Working  with  Others 

.03 

.04 

-.02 

.02 

.19 

-.03 

.24 

.06 

Overall  Performance 

.06 

.11 

.31 

.06 

.34 

-.01 

.19 

.07 

MOS-Specific 

All  MOS  Combined 

.00 

.07 

.36 

-.02 

- 

-.03 

.12 

.11 

1  IB 

.07 

.07 

- 

- 

- 

- 

- 

.14 

19K 

.36 

- 

.36 

- 

- 

- 

- 

.39 

25B 

- 

- 

- 

- 

- 

- 

- 

.20 

31B 

-.02 

- 

- 

-.02 

- 

- 

- 

.06 

68W 

-.03 

- 

- 

- 

- 

-.03 

- 

.04 

88M 

.12 

- 

- 

- 

- 

- 

.12 

.07 

Note.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  Army-wide  PRS:  All  MOS  Combined,  n  =  4,162-4,455;  1  IB,  n  =  1,430-1,433; 
19K,  n  =  90-94;  31B,  n  =  1,083-1,087;  42A,  n  =  99;  68W,  n  =  1,322-1,603;  88M,  n  =  93-97;  WTBD,  n  =  4,717-5,039.  MOS- 
specific  PRS:  All  MOS  Combined,  n  =  68-3,015;  1  IB,  n~  1,189;  19K,  n  =  68;  31B,  n=  973;  68W,n  =  671;  88M,  n  =  85; 
WTDB,  n  =  78-3,399.  Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 


Table  B.8.  Correlations  between  the  Job  Knowledge  Tests  (JKTs)  and  Performance  Rating 
Scales  (PRS)  in  the  In-Unit  Validation  Sample 


Domain/PRS 

In-Unit  JKT 

All  MOS 

Combined 

11B  19K 

31B  42A  68W  88M 

9  IB  WTBD 

Can  Do  a 

.14 

.20 

- 

.09 

Effort  &  Personal  Discipline  a 

.13 

.06 

-- 

.09 

Physical  Fitness  &  Bearing 

.06 

.07 

— 

.03 

Self-Management J 

.08 

.09 

-- 

.07 

Working  with  Others  a 

.16 

.11 

-- 

.11 

Overall  Leadership  Potential 

.11 

.15 

-- 

.05 

Note.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  Army-wide  PRS:  All  MOS  Combined,  n  =  250-259;  1  IB,  n  =  110-1 13;  WTDB, 
n  =  690-713.  Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 

“Ratings  composite  comprises  two  or  more  Army-wide  PRS. 


B-6 


Table  B.9.  Correlations  between  the  Job  Knowledge  Tests  (JKTs)  and  Administrative  Criteria 
in  the  IMT  Validation  Sample 

IMT  JKT 


All  MOS 


Domain/Measure 

Combined 

11B 

19K 

31B 

42A 

68W 

88M 

91B 

WTBD 

Attrition 

3 -Month  Cumulative 

-.06 

-.01 

.07 

-.05 

-.12 

-.01 

-.04 

-.11 

-.04 

6-Month  Cumulative 

-.06 

.01 

.03 

-.05 

-.12 

.02 

-.05 

-.11 

-.04 

9-Month  Cumulative 

-.02 

.03 

.03 

-.03 

-.07 

.04 

-.04 

-.11 

-.03 

12-Month  Cumulative 

-.01 

.04 

-.08 

.05 

-.19 

.05 

-.04 

-.11 

-.02 

15 -Month  Cumulative 

.01 

.06 

-.03 

.06 

-.19 

.07 

-.03 

-.12 

-.03 

18-Month  Cumulative 

.01 

.06 

-.03 

.07 

-.19 

.06 

.02 

-.12 

-.02 

2 1  -Month  Cumulative 

.03 

.07 

-.03 

.08 

-.19 

.06 

.04 

-.11 

.03 

24-Month  Cumulative 

.02 

.05 

.25 

.06 

-.19 

.07 

.05 

-.08 

.01 

27-Month  Cumulative 

.04 

.03 

.25 

.06 

-.19 

.08 

.04 

.11 

.03 

30-Month  Cumulative 

.05 

.01 

.25 

.06 

-.19 

.07 

.02 

.12 

.03 

Training  Restart 

Restarted  at  Least  Once  During  IMT 

-.05 

.02 

-.07 

.02 

.09 

-.04 

.00 

-.05 

.00 

Academic  or  Pejorative  Restart 

-.01 

.02 

-.07 

.01 

.09 

.02 

.00 

-.05 

.01 

Academic  Restart 

-.05 

.01 

-.07 

.02 

.09 

-.04 

.00 

-.05 

.00 

Final  AIT  School  Grades 


Overall  Average  (Unstandardized)  -.05  —  —  —  -  .17  -  —  .32 

Overall  Average  (Standardized) _ .27 _ ~ _ -- _ — _ -- _ AO _ -- _ -- _ .35 

Note.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  Attrition:  All  MOS  Combined,  n  =  13,031;  1  IB,  n  =  5,200;  19K,  n  =  280;  31B, 
n=  2,452;  42A,  n  =  216;  68W,  n  =  2,632;  88M,  n  =  1,997;  91B,  n  =  254;  WTBD,  n  =  15,520.  Attrition  Reasons:  All  MOS 
Combined,  n  =  1 1,676-1 1,828;  1  IB,  n  =  4,729-4,830;  19K,  n  =  219;  3  IB,  n  =  2,153-2,162;  68W,  n  =  2,477-2,492;  88M,  n  = 

1,71 1-1,726;  91B,  n  =  235-240;  WTDB,  n  =  13,935-14,122.  Training  Restart:  All  MOS  Combined,  n  =  9,324-9,457;  1  IB,  n  = 
3,850-3,865;  19K,  n  =  104;  31B,  n  =  1,804-1,815;  42A,  n  =  85;  68W,  n  =  1,668-1,748;  88M,  n  =  1,550-1,593;  91B,  n  =  242-247; 
WTBD,  n  =  1 1,096-1 1,254.  Final  AIT  School  Grade:  All  MOS  Combined,  n  =  98-99;  68W,  n  =  61;  WTBD,  n  =  250-252. 
Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 
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Table  B.10.  Correlations  between  the  Army  Life  Questionnaire  (ALQ)  and  Administrative  Criteria  in  the  IMT  Validation  Sample 


IMT  ALQ  Scale 


Domain/Measure 

AFF 

Army 

MOS 

NORM 

CAR 

RENL 

ATT 

AC 

AL 

DIS 

Last 

TRN 

TRN 

COM 

Fit 

Fit 

COM 

INT 

INT 

COG 

COM 

ADJ 

INC 

APFT 

ACH 

RES 

Attrition 

3 -Month  Cumulative 

.00 

-.01 

.12 

.02 

.01 

.00 

-.02 

.07 

-.03 

-.02 

-.04 

.02 

-.02 

6-Month  Cumulative 

.00 

-.01 

.11 

.02 

.00 

-.01 

-.01 

.07 

-.03 

-.03 

-.04 

.03 

-.02 

9-Month  Cumulative 

-.01 

-.01 

.08 

.01 

-.02 

-.03 

.00 

.05 

-.04 

-.03 

-.04 

.02 

-.01 

12-Month  Cumulative 

-.01 

-.02 

.07 

.01 

-.02 

-.03 

.00 

.04 

-.03 

-.03 

-.05 

.03 

-.02 

15-Month  Cumulative 

-.02 

-.02 

.06 

.00 

-.02 

-.03 

.00 

.02 

-.02 

-.03 

-.04 

.03 

-.02 

18-Month  Cumulative 

-.03 

-.03 

.06 

-.01 

-.03 

-.03 

.01 

.01 

-.03 

-.03 

-.04 

.02 

-.02 

21 -Month  Cumulative 

-.05 

-.04 

.04 

-.01 

-.04 

-.05 

.02 

.00 

-.04 

-.01 

-.04 

.01 

-.02 

24-Month  Cumulative 

-.04 

-.04 

.04 

-.01 

-.03 

-.04 

.01 

.01 

-.03 

-.01 

-.04 

.01 

-.02 

27-Month  Cumulative 

-.01 

-.01 

.05 

.02 

-.01 

-.02 

-.01 

.03 

-.01 

-.01 

-.03 

.01 

-.01 

30-Month  Cumulative 

-.02 

-.03 

.05 

.01 

-.01 

-.02 

.00 

.00 

-.02 

-.02 

-.02 

.01 

-.02 

Training  Restart 

Restarted  at  Least  Once  During  IMT 

.01 

.01 

.01 

.01 

.01 

.01 

-.04 

.03 

.03 

-.08 

.08 

-.02 

-.20 

Academic  or  Pejorative  Restart 

.00 

.02 

.02 

.02 

.00 

.00 

-.04 

.01 

.04 

-.09 

.09 

.03 

-.22 

Academic  Restart 

.00 

.00 

.00 

.00 

.00 

.00 

-.01 

.02 

.01 

-.07 

.06 

-.03 

-.20 

Final  AIT  School  Grades 

Overall  Average  (Unstandardized) 

-.03 

.02 

.07 

.04 

-.05 

-.07 

-.02 

-.08 

-.02 

.07 

-.07 

.09 

.04 

Overall  Average  (Standardized) 

-.08 

-.01 

.12 

.02 

-.09 

-.06 

-.04 

-.16 

.04 

.04 

-.12 

.01 

.05 

Note.  AFF  COM  =  Affective  Commitment;  NORM  COM  =  Nonnative  Commitment;  CAR  INT  =  Army  Career  Intentions;  RENL  INT  =  Army  Reenlistment  Intentions;  ATT 
COG  =  Attrition  Cognitions;  AC  COM  =  Army  Civilian  Comparison;  AL  ADJ  =  Army  Life  Adjustment;  DIS  INC  =  Disciplinary  Incidents  (#  of);  Last  APFT  =  Last  APFT  Score; 
TRN  ACH  =  Training  Achievements  (#  of);  TRN  RES  =  Training  Restart  (#  of).  Attrition:  n  =  1 4,65 1-1 6,243 .  Attrition  Reason:  n  =  13,109-14,781.  Training  Restart:  n  =  10,464- 
1 1,786.  Final  AIT  School  Grade:  n  =  157-265.  Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 
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Table  B.l  1.  Correlations  between  the  Performance  Rating  Scales  ( PRS )  and  Attrition  in  the  IMT  Validation  Sample 


Attrition 

Domain/PRS 

3- 

Month 

6- 

Month 

9- 

Month 

12- 

Month 

15- 

Month 

18- 

Month 

21- 

Month 

24- 

Month 

27- 

Month 

30- 

Month 

Army-  Wide 

Adjustment  to  the  Army 

.01 

-.03 

-.06 

-.09 

-.11 

-.11 

-.09 

-.10 

-.09 

-.08 

Effort  &  Discipline 

-.01 

-.05 

-.08 

-.13 

-.15 

-.14 

-.11 

-.10 

-.11 

-.08 

MOS  Qualification  Knowledge  &  Skill 

-.01 

-.04 

-.06 

-.10 

-.12 

-.11 

-.08 

-.09 

-.09 

-.07 

Physical  Fitness  &  Bearing 

.01 

-.02 

-.05 

-.10 

-.12 

-.11 

-.09 

-.10 

-.10 

-.08 

Working  with  Others 

.00 

-.05 

-.10 

-.16 

-.19 

-.18 

-.13 

-.12 

-.12 

-.10 

Overall  Performance 

.00 

-.02 

.01 

-.01 

.01 

.01 

.02 

-.01 

-.01 

-.01 

MOS-Specific 

All  MOS  Combined 

.00 

-.01 

-.06 

-.14 

-.16 

-.17 

-.12 

-.12 

-.11 

-.09 

(1 1B/C/X  and  18X) 

.02 

.02 

-.07 

-.13 

-.14 

-.14 

-.02 

-.04 

-.05 

-.03 

19K 

-.11 

-.11 

-.11 

-.29 

-.37 

-.37 

-.37 

-.11 

-.11 

-.11 

25B 

-.10 

-.10 

-.11 

-.10 

-.10 

-.10 

-.10 

-.10 

-.10 

-.10 

31B 

-.01 

.00 

-.03 

-.08 

-.10 

-.16 

-.11 

-.17 

-.10 

-.05 

68W 

.01 

-.06 

-.08 

-.24 

-.25 

-.24 

-.23 

-.23 

-.21 

-.15 

88M 

.19 

.19 

.19 

.21 

.20 

.19 

.19 

.28 

.25 

.15 

Note.  Army-wide  PRS:  Attrition,  n  =  4,853-5,177;  Attrition  Reason,  n  =4,393-4,751.  MOS-specific  PRS:  Attrition,  n  =  85-3,507;  Attrition  Reason,  n  =  70-3,248.  Correlations  in 
bold  are  statistically  significant  (p  <  .05,  two-tailed). 


Table  B.12.  Correlations  between  the  Performance  Rating  Scales  (PRS)  and  Administrative 
Criteria  in  the  IMT  Validation  Sample 


Training  Restart 

Domain/PRS 

IMT 

Restart 

PEJ 

Restart 

ACAD 

Restart 

Army-  Wide  (IMT) 

Adjustment  to  the  Army 

.05 

.05 

.04 

Effort  &  Discipline 

.00 

.01 

-.01 

MOS  Qualification  Knowledge  &  Skill 

.01 

.02 

.01 

Physical  Fitness  &  Bearing 

.02 

.04 

.00 

Working  with  Others 

.00 

.00 

-.01 

Overall  Performance 

.03 

.05 

.01 

MOS-Specific  (IMT) 

All  MOS  Combined 

.00 

.01 

.00 

11B 

-.02 

-.02 

-.02 

25B 

.13 

.13 

.01 

31B 

.05 

.05 

.03 

68W 

-.03 

-.04 

-.03 

88M 

-.22 

.06 

-.22 

Note.  IMT  Restart  =  Restarted  at  Least  Once  During  IMT;  PEJ  Restart  =  Restarted  at  Least  Once  for  Academic  or  Other 
Pejorative  Reason;  ACAD  Restart  =  Restarted  at  Least  Once  for  Academic  Reasons  Army-wide  PRS:  Training  Restart,  n  = 
3,684-3,929.  MOS-Specific  PRS:  Training  Restart,  n  =  67-2,825;  Final  AIT  Grade,  n  =  141-143.  Correlations  in  bold  are 
statistically  significant  (p  <  .05,  two-tailed). 
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Table  B.13.  Correlations  among  Criterion  Composites  in  the  IMT  and  In-Unit  Validation  Samples 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13  14 

1 

IMT :  Overall  Performance 

2 

IMT:  Physical  Fitness 

.19 

3 

IMT:  Commitment  &  Fit 

.07 

.08 

4 

IMT :  Retention  Cognitions 

.01 

.01 

.36 

5 

IMT:  Knowledge  &  Skill 

.08 

.05 

.15 

-.05 

6 

IMT:  Training  Restart  (Y/N) 

-.05 

-.24 

-.07 

-.01 

-.06 

7 

IMT:  Disciplinary  Incidents  (Y/N) 

-.10 

-.16 

-.07 

-.04 

-.02 

.18 

8 

IMT:  Army  Life  Adjustment 

.09 

.22 

.55 

.26 

.13 

-.17 

-.16 

9 

IU :  Overall  Performance 

- 

.15 

-.06 

-.13 

.10 

-.11 

-.22 

-.01 

10 

IU:  Physical  Fitness 

.06 

.57 

.03 

.10 

.01 

-.09 

.02 

.07 

.19 

11 

IU :  Commitment  &  Fit 

.00 

.00 

.46 

.21 

.07 

-.07 

-.15 

.30 

.06 

.06 

12 

IU :  Retention  Cognitions 

.15 

-.01 

.27 

.49 

-.01 

-.05 

-.07 

.18 

.01 

.04 

.42 

13 

IU :  Knowledge  &  Skill 

-.13 

.08 

.24 

-.14 

.50 

.00 

.02 

.07 

.12 

-.01 

.09 

-.01 

14 

IU:  Leadership  Potential 

- 

.20 

.02 

-.11 

.14 

-.14 

-.33 

.07 

.75 

.19 

.09 

.01 

.07 

15 

IU:  Disciplinary  Incidents  (Y/N) 

.09 

-.14 

.00 

.12 

-.09 

-.01 

.12 

-.06 

-.23 

-.09 

-.15 

-.05 

-.05  -.26 

Note.  Bolded  values  indicated  correlation  is  significant  at  the  0.05  level  (2 -tailed).  Statistics  based  on  fewer  than  50  cases  are  not  reported.  Will-Do  includes  the  performance 
ratings  scales,  APFT,  the  ALQ  scales,  training  achievement,  Training  Restarts,  and  disciplinary  incidents.  Sample  sizes  range  from  46-15,294. 
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CRITERION  PSYCHOMETRIC  PROPERTIES  IN  THE  FULL  IMT  AND  IN-UNIT 

SAMPLES 


Table  C.  1 .  Descriptive  Statistics  and  Reliability  Estimates  for  the  Job  Knowledge  Tests  (JKTs) 
in  the  Full  IMT  and  In-Unit  Samples 


Domain/Setting/JKT 

n 

M 

SD 

Min 

Max 

t'wTBD 

a 

IMT 

MOS-Specific 

11B/C/X  +  18X 

14,384 

61.20 

10.15 

20.93 

88.37 

.55 

.78 

19K 

534 

61.88 

11.67 

20.29 

85.71 

.46 

.73 

31B 

6,845 

68.29 

8.54 

34.95 

93.20 

.50 

.77 

42A 

360 

56.50 

12.35 

25.93 

81.48 

.50 

.75 

68W 

8,620 

73.39 

10.34 

25.00 

96.74 

.49 

.87 

88M 

5,786 

63.77 

10.60 

30.56 

94.44 

.54 

.77 

91B 

1,187 

57.34 

13.47 

23.71 

90.72 

.47 

.90 

All  MOS  Combined 

37,716 

65.51 

11.38 

20.29 

96.74 

.54 

WTBD  (Army- Wide) 

45,911 

64.81 

12.55 

6.45 

100.00 

- 

.65 

In-Unit 

MOS-Specific 

11B/C/X  +  18X 

612 

63.19 

9.86 

26.76 

84.51 

.57 

.71 

19K 

59 

75.63 

11.82 

37.25 

90.74 

.45 

.82 

31B 

81 

62.29 

11.37 

33.64 

81.31 

.59 

.88 

42A 

45 

51.92 

5.90 

38.82 

64.71 

.16 

- 

68W 

147 

72.26 

8.44 

48.11 

90.57 

.49 

.65 

88M 

197 

64.16 

9.70 

40.43 

87.23 

.59 

.79 

91B 

111 

61.92 

11.78 

35.09 

85.26 

.29 

.81 

All  MOS  Combined 

1,318 

64.29 

10.97 

26.76 

90.74 

.50 

WTBD  (Army-Wide) 

3,679 

65.21 

12.47 

15.38 

100.00 

- 

.59 

Note.  M,  SD ,  Min,  and  Max  are  based  on  percent  correct;  a  =  coefficient  alpha.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  7‘wtbd 
=  correlation  with  WTBD  JKT  scores.  All  correlations  are  statistically  significant  (p  <  .05). 
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Table  C.2.  Descriptive  Statistics  and  Reliability  Estimates  for  the  Performance  Rating  Scales 
(PRS)  in  the  Full  IMT  and  In-Unit  Samples 


Domain/Setting/PRS 

n 

M 

SD 

Min 

Max 

a 

IRR 

IMT 

Army-  Wide 

Adjustment  to  the  Army 

16,174 

3.25 

1.02 

1.00 

5.00 

- 

.15 

Effort  &  Personal  Discipline 

16,202 

3.05 

.99 

1.00 

5.00 

~ 

.18 

MOS  Qualification  Knowledge 

14,699 

3.20 

.96 

1.00 

5.00 

- 

.10 

Physical  Fitness  &  Bearing 

16,103 

3.10 

1.01 

1.00 

5.00 

- 

.19 

Working  with  Others 

16,145 

3.02 

.99 

1.00 

5.00 

- 

.15 

Overall  Performance 

15,960 

3.51 

.84 

1.00 

5.00 

- 

.31 

MOS-Specific 

11B/C/X  +  18X 

4,919 

2.99 

.81 

1.00 

5.00 

.96 

.17 

19K 

194 

3.28 

.56 

1.57 

4.86 

.91 

.41 

31B 

2,601 

3.20 

.78 

1.00 

5.00 

.97 

.10 

68W 

3,465 

2.71 

.77 

1.00 

5.00 

.95 

.01 

88M 

673 

2.87 

.76 

1.20 

5.00 

.93 

.00 

91B 

255 

2.97 

1.90 

1.00 

5.00 

.97 

.11 

All  MOS  Combined 

12,070 

2.95 

.81 

1.00 

5.00 

In-Unit 

Army-  Wide 

Can  Do  a 

2,690 

4.89 

1.27 

1.00 

7.00 

.84 

— 

Effort  &  Personal  Discipline  a 

2,688 

5.21 

1.37 

1.00 

7.00 

- 

- 

Physical  Fitness  &  Bearing 

2,678 

5.26 

1.58 

1.00 

7.00 

- 

- 

Self-Management a 

2,682 

5.30 

1.14 

1.00 

7.00 

- 

- 

Working  with  Others  a 

2,689 

5.29 

1.22 

1.00 

7.00 

.75 

- 

Overall  Leadership  Potential 

2,637 

4.76 

1.67 

1.00 

7.00 

- 

- 

Note.  Ratings  on  IMT  PRS  range  from  1  and  5,  except  for  the  Overall  Performance  PRS,  which  ranges  from  1  to  5.  PRS  ratings 
from  supervisors  with  a  familiarity  rating  of  1  (“I  have  had  little  opportunity  to  observe  this  Soldier”)  were  excluded  from 
analyses,  a  =  coefficient  alpha.  IRR  =  Interrater  reliability,  estimated  using  G(q,k)  (Putka,  Le,  McCloy,  &  Diaz,  2008).  IRR 
estimates  were  not  estimated  if  30  or  fewer  Soldiers  were  rated  by  more  than  one  supervisor. 
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Table  C.3.  Descriptive  Statistics  and  Reliability  Estimates  for  the  Army  Life  Questionnaire 
(ALQ)  in  the  Full  IMT  and  In-Unit  Samples 


Domain/Setting/Scale 

n 

M 

SD 

Min 

Max 

a 

IMT 

Affective  Commitment 

47,913 

3.89 

.68 

1.00 

5.00 

.86 

Army  Fit 

47,913 

4.08 

.60 

1.00 

5.00 

.86 

MOS  Fit 

47,913 

3.78 

.85 

1.00 

5.00 

.93 

Normative  Commitment 

47,913 

4.17 

.70 

LOO 

5.00 

.79 

Army  Career  Intentions 

47,913 

3.19 

1.10 

1.00 

5.00 

.91 

Army  Reenlistment  Intentions 

47,913 

3.48 

.96 

1.00 

5.00 

.83 

Attrition  Cognition 

47,913 

1.52 

.61 

1.00 

5.00 

.77 

Army  Life  Adjustment 

47,913 

4.08 

.66 

1.00 

5.00 

.87 

Disciplinary  Incidents  (#) 

35,668 

.27 

.62 

.00 

7.00 

- 

Last  APFT  Score 

47,418 

250.59 

31.89 

10.00 

300.00 

- 

Training  Achievement 

47,870 

.40 

.61 

.00 

2.00 

- 

Training  Restart  (#) 

47,911 

.40 

.64 

.00 

4.00 

- 

In-Unit 

Affective  Commitment 

3,730 

3.61 

.80 

LOO 

5.00 

.89 

Army  Fit 

3,730 

3.90 

.71 

LOO 

5.00 

.81 

MOS  Fit 

3,730 

3.24 

.95 

1.00 

5.00 

.93 

MOS  Satisfaction 

3,730 

3.53 

.91 

1.00 

5.00 

.93 

Army  Career  Intentions 

3,730 

2.67 

1.19 

LOO 

5.00 

.93 

Army  Reenlistment  Intentions 

3,729 

3.07 

1.17 

1.00 

5.00 

.79 

Attrition  Cognition 

3,730 

1.70 

.74 

1.00 

5.00 

.77 

Disciplinary  Incidents  (#) 

3,730 

.39 

.90 

.00 

7.00 

- 

Last  APFT  Score 

3,640 

245.36 

33.50 

1.00 

300.00 

- 

Note,  a  =  coefficient  alpha. 
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Table  C.4.  Correlations  among  the  Performance  Rating  Scales  (PRS)  in  the  Full  IMT Sample 


Domain/PRS 

1 

2 

3 

4 

5 

6 

Army-  Wide 

1. 

Adjustment  to  the  Army 

2. 

Effort  &  Personal  Discipline 

.77 

3. 

MOS  Qualification  Knowledge 

.71 

.68 

4. 

Physical  Fitness  &  Bearing 

.69 

.71 

.62 

5. 

Working  with  Others 

.74 

.75 

.70 

.66 

6. 

Overall  Performance 

.56 

.57 

.53 

.54 

.54 

MOS-Specific 

7. 

11B/C/X+  18X 

.67 

.64 

.70 

.61 

.66 

.53 

8. 

19K 

.66 

.68 

.76 

.61 

.42 

.58 

9. 

31B 

.64 

.63 

.69 

.53 

.65 

.56 

10. 

68W 

.55 

.50 

.61 

.46 

.56 

.33 

11. 

88M 

.59 

.54 

.64 

.55 

.57 

.50 

12. 

91B 

.72 

.67 

.80 

.67 

.73 

.58 

13. 

All  MOS  Combined 

.64 

.60 

.68 

.56 

.63 

.48 

Note.  Army-wide  PRS:  n  =  14,612-16,202.  MOS-specific  PRS:  1  IB,  n  =  4,527-4,533;  19K,  n  =  185;  31B,  n  =  2,446-2,462;  68W, 
n  =  2,071-2,608;  88M,  n  =  608-625;  91B,  n  =  227-246;  All  MOS  Combined,  n  =  10,082-10,626.  Ratings  on  PRS  range  from  1 
and  5.  PRS  ratings  from  supervisors  with  a  familiarity  rating  of  1  (“I  have  had  little  opportunity  to  observe  this  Soldier”)  were 
excluded  from  analyses.  All  correlations  are  statistically  significant  (p  <  .05,  one-tailed). 


Table  C.5.  Correlations  among  Performance  Rating  Scales  (PRS)  in  the  Full  In-Unit  Sample 


Domain/PRS 

1 

2 

3 

4 

5 

1. 

Can  Do  a 

2. 

Effort  &  Personal  Discipline  a 

.78 

3. 

Physical  Fitness  &  Bearing 

.58 

.61 

4. 

Self-Management a 

.76 

.75 

.61 

5. 

Working  with  Others  a 

.78 

.77 

.58 

.75 

6. 

Overall  Leadership  Potential 

.68 

.70 

.61 

.67 

.64 

Note.  Army-wide  PRS,  n  =  2,625-2,690.  Ratings  on  PRS  range  from  1  to  7.  PRS  ratings  from  supervisors  with  a  familiarity  rating 
of  1  (“I  have  had  little  opportunity  to  observe  this  Soldier”)  were  excluded  from  analyses.  All  correlations  are  statistically 
significant  (p  <  .05,  one-tailed). 

a  Ratings  composite  comprises  two  or  more  Army-wide  PRS. 
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Table  C.6.  Correlations  among  Army  Life  Questionnaire  (ALQ)  Scales  in  the  Full  IMT  and  In-Unit  Samples 


Domain/Scale 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

1. 

Affective  Commitment 

.78 

.39 

.51 

- 

.60 

.53 

-.58 

— 

-.15 

.06 

— 

2. 

Army  Fit 

.84 

.42 

.53 

- 

.59 

.56 

-.66 

- 

-.20 

.09 

- 

3. 

MOS  Fit 

.48 

.48 

.56 

- 

.24 

.18 

-.31 

- 

-.12 

.03 

- 

4. 

MOS  Satisfaction 

— 

— 

— 

— 

.35 

.28 

-.38 

— 

-.14 

.03 

— 

5. 

Normative  Commitment 

.69 

.71 

.41 

— 

— 

— 

— 

- 

— 

— 

- 

6. 

Army  Career  Intentions 

.56 

.54 

.25 

— 

.43 

.79 

-.49 

- 

-.11 

.05 

— 

7. 

Army  Reenlistment  Intentions 

.54 

.54 

.28 

- 

.46 

.85 

-.44 

- 

-.08 

.05 

- 

8. 

Attrition  Cognition 

-.63 

-.69 

-.41 

- 

-.74 

-.47 

-.50 

- 

.23 

-.12 

- 

9. 

Army  Life  Adjustment 

.45 

.61 

.35 

- 

.45 

.36 

.40 

-.53 

- 

- 

- 

10. 

Disciplinary  Incidents  (#) 

-.07 

-.10 

-.08 

- 

-.07 

-.04 

-.05 

.11 

-.17 

-.04 

- 

11. 

Last  APFT  Score 

.05 

.10 

.07 

- 

.07 

.03 

.04 

-.12 

.24 

-.14 

- 

12. 

Training  Achievement 

.06 

.07 

.05 

- 

.00 

.09 

.07 

-.04 

.13 

-.07 

.23 

13. 

Training  Restart  (#) 

-.06 

-.09 

-.08 

- 

-.07 

-.03 

-.04 

.11 

-.20 

.20 

-.27 

-.12 

Note.  Correlations  below  the  diagonal  are  based  on  the  Full  IMT  sample,  n  =  35,336  -  47,913.  Correlations  above  the  diagonal  are  based  on  the  Full  In-Unit  sample,  n  =  3,639  -  3,730. 


Correlations  in  bold  are  statistically  significant  (p  <  .05,  two-tailed). 
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APPENDIX  D 


SUMMARY  OF  BIVARIATE  CORRELATIONS  BETWEEN  TAPAS  SCALES  AND  SELECTED  CRITERIA 


Table  D.l.  Summary  of  the  Bivariate  Correlations  between  AFQT,  TAPAS,  and  Selected  IMT  Criteria  for  Tier  1  Soldiers 


WTBD 

JKT 

MOS- 

Speciftc 

JKT 

Final  AIT 
Grade 

Effort  & 
Personal 
Discipline 
(PRS) 

Disciplinary 

Incidents 

(ALQ) 

Training 

Restart 

(ALQ) 

Last 

APFT 

Score 

(ALQ) 

Army  Life 
Adjustment 
(PRS) 

Army  Life 
Adjustment 
(ALQ) 

Army  Fit 
(ALQ) 

AFQT 

.37 

.37 

.29 

.09 

-.02 

.00 

.09 

.04 

.07 

-.04 

Individual  TAPAS  Scales 
Achievement 

.03 

.04 

.07 

.07 

-.07 

.01 

.09 

.06 

.14 

.12 

Adjustment 

.05 

.06 

.00 

-.03 

-.02 

-.01 

.01 

-.01 

.09 

.02 

Adventure  Seeking  a 

.09 

.05 

.03 

.06 

-.03 

.01 

.06 

-.03 

.05 

.00 

Attention  Seeking 

.01 

.00 

.00 

.01 

-.01 

.01 

.07 

.01 

.07 

.03 

Commitment  to  Serve  a 

-.06 

-.01 

-.02 

-.02 

.03 

-.03 

-.04 

-.02 

.14 

.18 

Cooperation 

-.01 

-.01 

.00 

-.01 

.00 

.01 

-.01 

.00 

.00 

.01 

Courage  a 

.06 

.05 

.01 

-.09 

-.05 

-.03 

.10 

.00 

.15 

.14 

Dominance 

.04 

.01 

.04 

.04 

-.05 

.01 

.12 

.04 

.13 

.10 

Even  Tempered 

.04 

.03 

.02 

.01 

-.02 

-.01 

-.05 

.00 

.03 

.02 

Intellectual  Efficiency 

.16 

.16 

.11 

.02 

-.02 

-.02 

.03 

.01 

.11 

.03 

Non-Delinquency 

.00 

-.02 

.02 

.00 

-.03 

.00 

-.04 

-.01 

.01 

.05 

Optimism 

-.01 

.00 

.01 

.03 

-.04 

.01 

.04 

.04 

.10 

.06 

Order 

-.07 

-.06 

-.02 

.01 

-.01 

.00 

.03 

-.01 

.00 

.01 

Physical  Conditioning 

.01 

-.02 

.00 

.07 

-.11 

.05 

.28 

.07 

.13 

.04 

Responsibility  J 

-.01 

.04 

.06 

.07 

-.04 

.00 

.03 

.03 

.08 

.04 

Self-Control 

.01 

.01 

.02 

.02 

-.02 

-.01 

.00 

.01 

.03 

.04 

Selflessness 

-.02 

-.04 

-.01 

-.01 

.01 

-.01 

-.01 

-.01 

-.01 

.05 

Situational  Awareness  a 

.09 

.07 

.01 

.06 

.00 

-.05 

-.02 

.01 

.08 

.08 

Sociability 

-.08 

-.09 

-.06 

-.01 

.01 

.01 

.04 

.00 

.04 

.04 

Team  Orientation  a 

.00 

-.05 

-.04 

-.02 

-.02 

-.03 

-.04 

-.05 

-.02 

.03 

Tolerance 

-.01 

-.02 

-.03 

.00 

.02 

-.01 

.00 

.00 

.02 

.05 

TOPS  Composites  (Original) 
Can-Do 

.09 

.09 

.09 

.04 

-.07 

.00 

.03 

.02 

.15 

.10 

Will-Do 

.04 

.01 

.04 

.07 

-.09 

.02 

.09 

.05 

.10 

.09 

TOPS  Composites  (Revised) 
Can-Do 

.20 

.21 

.15 

.02 

-.03 

-.01 

.00 

.01 

.09 

.01 

Will-Do 

.03 

.01 

.04 

.09 

-.12 

.04 

.26 

.09 

.20 

.12 

Adaptation 

.09 

.06 

.04 

.06 

-.09 

.04 

.17 

.05 

.10 

.03 

Note.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed). Correlations  based  on  fewer  than  50  cases  are  not  reported.  Sample  sizes  range  from  n  =  4,688  -  36,420 
JKT  =  Job  Knowledge  Test.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  PRS  =  Performance  Rating  Scales.  ALQ  =  Army  Life  Questionnaire. 


"  Sample  sizes  for  six  TAPAS  scales  were  considerably  smaller  than  the  other  dimensions  ( n  =  279  -  2,985 
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Table  D.2.  Summary  of  the  Bivariate  Correlations  between  AFQT,  TAPAS,  and  Selected  IMT  Criteria  for  Tier  2  Soldiers 


WTBD 

JKT 

MOS- 

Speciftc 

JKT 

Final  AIT 
Grade 

Effort  & 
Personal 
Discipline 
(PRS) 

Disciplinary 

Incidents 

(ALQ) 

Training 

Restart 

(ALQ) 

Last 

APFT 

Score 

(ALQ) 

Army  Life 
Adjustment 
(PRS) 

Army  Life 
Adjustment 
(ALQ) 

Army 

Fit 

(ALQ) 

AFQT 

.29 

.24 

.26 

.02 

-.03 

-.03 

.06 

-.02 

.01 

-.02 

Individual  TAPAS  Scales 
Achievement 

.09 

.01 

.07 

.03 

-.04 

.00 

.14 

.00 

.13 

.10 

Adjustment 

.07 

.05 

.08 

-.14 

.06 

.00 

.04 

-.14 

.03 

-.01 

Adventure  Seeking  a 

.05 

.12 

.00 

- 

.02 

-.02 

.17 

- 

-.02 

-.26 

Attention  Seeking 

-.04 

-.01 

-.04 

.05 

.00 

.02 

.06 

.06 

.07 

.04 

Commitment  to  Serve  d 

.20 

.24 

.01 

- 

.01 

-.14 

.02 

- 

.17 

.14 

Cooperation 

-.01 

-.06 

.01 

-.03 

.03 

-.03 

-.04 

.08 

-.05 

-.03 

Courage  a 

.03 

.02 

.04 

- 

.16 

-.02 

.06 

- 

.20 

.12 

Dominance 

.03 

.00 

.01 

.03 

-.03 

-.01 

.17 

.05 

.12 

.14 

Even  Tempered 

.04 

.03 

.05 

-.02 

.05 

-.01 

-.05 

-.04 

.01 

-.05 

Intellectual  Efficiency 

.07 

.05 

.13 

.04 

-.01 

-.02 

.04 

-.05 

.08 

.08 

Non-Delinquency 

.07 

-.02 

.02 

-.06 

.01 

-.01 

-.01 

-.04 

.03 

.06 

Optimism 

-.03 

-.07 

.01 

-.02 

.01 

-.01 

.08 

.13 

.09 

.06 

Order 

-.02 

-.06 

.01 

.08 

.00 

.00 

.00 

-.02 

.06 

.03 

Physical  Conditioning 

-.07 

-.01 

.02 

.11 

-.05 

.01 

.22 

.11 

.11 

.07 

Responsibility 

.08 

.06 

.05 

- 

-.03 

.14 

-.06 

- 

.14 

.40 

Self-Control 

.08 

-.04 

.03 

-.05 

-.06 

-.05 

.01 

-.04 

-.05 

.00 

Selflessness 

-.05 

-.01 

-.05 

.00 

.04 

-.02 

.04 

.00 

-.07 

-.06 

Situational  Awareness 

-.03 

-.10 

.04 

— 

-.02 

-.06 

.00 

— 

.09 

-.15 

Sociability 

.01 

.00 

-.04 

.00 

.01 

.01 

.08 

.19 

.09 

.12 

Team  Orientation  a 

-.09 

-.11 

.00 

— 

-.15 

-.05 

.20 

- 

.23 

.22 

Tolerance 

-.04 

-.05 

-.04 

-.06 

-.07 

-.01 

.08 

-.05 

.00 

.00 

TOPS  Composites  (Original) 
Can-Do 

.09 

.00 

.11 

-.05 

.03 

-.02 

.09 

-.03 

.12 

.09 

Will-Do 

.09 

.01 

.08 

.02 

-.02 

.00 

.11 

.00 

.09 

.07 

TOPS  Composites  (Revised) 
Can-Do 

.10 

.07 

.19 

.03 

-.04 

-.02 

.02 

-.10 

.03 

.05 

Will-Do 

-.01 

-.02 

.05 

.09 

-.06 

.00 

.27 

.13 

.18 

.14 

Adaptation 

-.03 

.01 

.03 

.03 

-.02 

-.01 

.14 

.00 

.07 

-.02 

Note.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed).Correlations  based  on  fewer  than  50  cases  are  not  reported.  Sample  sizes  range  from  n  =  109  -  2,404 
"  Sample  sizes  for  six  TAPAS  dimensions  were  considerably  smaller  than  the  other  dimensions  (n  =  22  -  229).  JKT  =  Job  Knowledge  Test.  WTBD  =  Warrior  Tasks  and  Battle 
Drills.  PRS  =  Perfonnance  Rating  Scales.  ALQ  =  Army  Life  Questionnaire. 
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Table  D.3.  Summary  of  the  Bivariate  Correlations  between  AFQT,  TAPAS,  and  Selected  IMT  Criteria  for  Tier  1+2  Soldiers 


WTBD 

JKT 

MOS- 

Speciftc 

JKT 

Final  AIT 
Grade 

Effort  & 
Personal 
Discipline 
(PRS) 

Disciplinary 

Incidents 

(ALQ) 

Training 

Restart 

(ALQ) 

Last 

APFT 

Score 

(ALQ) 

Army  Life 
Adjustment 
(PRS) 

Army  Life 
Adjustment 
(ALQ) 

Army  Fit 
(ALQ) 

AFQT 

.37 

.36 

.29 

.08 

-.02 

.00 

.09 

.04 

.07 

-.04 

Individual  TAPAS  Scales 
Achievement 

.04 

.04 

.07 

.07 

-.07 

.01 

.09 

.06 

.14 

.12 

Adjustment 

.05 

.06 

.00 

-.03 

-.02 

-.01 

.01 

-.01 

.09 

.02 

Adventure  Seeking  a 

.09 

.05 

.02 

.08 

-.02 

.01 

.07 

-.01 

.05 

-.02 

Attention  Seeking 

.01 

.00 

.00 

.01 

-.01 

.01 

.07 

.01 

.07 

.03 

Commitment  to  Serve  a 

-.05 

.01 

-.02 

-.02 

.03 

-.04 

-.04 

-.02 

.14 

.18 

Cooperation 

-.01 

-.01 

.00 

-.01 

.00 

.00 

-.01 

.01 

.00 

.00 

Courage  a 

.06 

.05 

.02 

-.04 

-.03 

-.03 

.09 

.00 

.16 

.14 

Dominance 

.03 

.01 

.04 

.04 

-.05 

.01 

.12 

.04 

.13 

.10 

Even  Tempered 

.04 

.03 

.02 

.01 

-.01 

-.01 

-.05 

.00 

.03 

.02 

Intellectual  Efficiency 

.16 

.16 

.11 

.02 

-.02 

-.02 

.03 

.00 

.11 

.03 

Non-Delinquency 

.00 

-.02 

.02 

-.01 

-.03 

.00 

-.04 

-.01 

.01 

.05 

Optimism 

-.01 

-.01 

.01 

.03 

-.04 

.01 

.04 

.04 

.10 

.06 

Order 

-.07 

-.06 

-.02 

.01 

-.01 

.00 

.03 

-.01 

.00 

.01 

Physical  Conditioning 

.00 

-.02 

.00 

.07 

-.10 

.05 

.28 

.07 

.13 

.04 

Responsibility 

.00 

.04 

.06 

.07 

-.04 

.01 

.02 

.02 

.09 

.07 

Self-Control 

.02 

.01 

.02 

.02 

-.02 

-.01 

.00 

.01 

.03 

.04 

Selflessness 

-.02 

-.04 

-.01 

-.01 

.01 

-.01 

-.01 

-.01 

-.01 

.04 

Situational  Awareness 

.09 

.06 

.01 

.05 

.01 

-.05 

-.02 

.01 

.09 

.07 

Sociability 

-.08 

-.09 

-.06 

-.01 

.01 

.01 

.04 

.00 

.05 

.04 

Team  Orientation  a 

.00 

-.05 

-.04 

.02 

-.03 

-.04 

-.02 

-.02 

.01 

.04 

Tolerance 

-.01 

-.02 

-.03 

.00 

.01 

-.01 

.00 

.00 

.02 

.05 

TOPS  Composites  (Original) 
Can-Do 

.09 

.09 

.09 

.04 

-.06 

.00 

.03 

.02 

.15 

.10 

Will-Do 

.04 

.01 

.04 

.07 

-.08 

.02 

.09 

.05 

.10 

.09 

TOPS  Composites  (Revised) 
Can-Do 

.20 

.20 

.15 

.02 

-.03 

-.01 

.00 

.01 

.09 

.01 

Will-Do 

.02 

.01 

.04 

.09 

-.12 

.04 

.26 

.09 

.20 

.12 

Adaptation 

.08 

.06 

.04 

.05 

-.09 

.03 

.17 

.05 

.10 

.03 

Note.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed). Correlations  based  on  fewer  than  50  cases  are  not  reported.  Sample  size  ranges  from  n  =  4,230  -  66,172 
°  Sample  sizes  for  six  TAPAS  dimensions  were  considerably  smaller  than  the  other  dimensions  (n  =  301  -  3,192).  JKT  =  Job  Knowledge  Test.  WTBD  =  Warrior  Tasks  and  Battle 
Drills.  PRS  =  Performance  Rating  Scales.  ALQ  =  Army  Life  Questionnaire. 
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Table  D.4.  Summary  of  the  Bivariate  Correlations  between  AFQT,  TAPAS,  and  Selected  In-Unit  Criteria  for  Tier  1  Soldiers 


WTBD 

JKT 

Can-Do 

(PRS) 

Effort  & 
Personal 
Discipline 
(PRS) 

Disciplinary 

Incidents 

(ALQ) 

Last 

APFT 

Score 

(ALQ) 

Working 

with 

Others 

(PRS) 

Overall 

Leadership 

(PRS) 

Affective 

Commitment 

(ALQ) 

Army 

Fit 

(ALQ) 

Reenlistment 

Intention 

(ALQ) 

AFQT 

.37 

.11 

.12 

-.02 

-.02 

.13 

.07 

-.06 

-.05 

-.09 

Individual  TAPAS  Scales 
Achievement 

.04 

.11 

.11 

-.09 

.03 

.09 

.08 

.08 

.10 

.11 

Adjustment 

.07 

.01 

.00 

-.03 

-.01 

.00 

-.02 

.01 

.04 

.01 

Adventure  Seeking  a 

- 

- 

- 

- 

- 

- 

- 

— 

- 

— 

Attention  Seeking 

.00 

-.01 

-.03 

.06 

-.01 

-.06 

-.03 

-.02 

.03 

-.07 

Commitment  to  Serve  a 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

Cooperation 

-.01 

-.05 

-.09 

-.06 

-.04 

-.06 

-.06 

.04 

.06 

.03 

Courage  a 

- 

- 

- 

- 

- 

- 

- 

— 

- 

— 

Dominance 

.03 

.05 

.03 

.00 

.08 

.03 

.02 

.06 

.08 

.05 

Even  Tempered 

.10 

.00 

-.06 

-.04 

-.04 

.00 

-.01 

.07 

.07 

.07 

Intellectual  Efficiency 

.16 

.06 

.03 

-.03 

.00 

.05 

-.04 

.05 

.05 

.01 

Non-Delinquency 

.00 

.07 

.02 

-.05 

-.08 

.05 

.04 

.08 

.06 

.07 

Optimism 

.00 

.02 

.01 

-.03 

.01 

.02 

.01 

.03 

.07 

.03 

Order 

-.08 

-.05 

-.03 

-.08 

.05 

-.01 

-.03 

.04 

.03 

.06 

Physical  Conditioning 

.00 

.06 

.06 

-.05 

.26 

.08 

.07 

.00 

.00 

-.01 

Responsibility 

- 

- 

- 

- 

- 

- 

- 

— 

- 

— 

Self-Control 

.04 

.02 

.04 

-.06 

.02 

.01 

.02 

.07 

.04 

.08 

Selflessness 

-.06 

-.04 

-.02 

.00 

-.04 

-.03 

-.02 

.04 

.03 

.04 

Situational  Awareness 

— 

— 

— 

- 

— 

— 

- 

— 

— 

— 

Sociability 

-.09 

.02 

-.02 

.03 

.04 

.00 

-.01 

.03 

.02 

.01 

Team  Orientation  a 

— 

— 

— 

— 

— 

— 

- 

— 

— 

— 

Tolerance 

-.03 

.00 

.03 

.04 

-.01 

.04 

.02 

.03 

.05 

.02 

TOPS  Composites  (Original) 
Can-Do 

.11 

.10 

.04 

-.08 

-.02 

.08 

.03 

.11 

.13 

.11 

Will-Do 

.05 

.11 

.07 

-.11 

.09 

.12 

.09 

.10 

.09 

.14 

TOPS  Composites  (Revised) 
Can-Do 

.22 

.07 

.04 

-.03 

-.04 

.05 

-.01 

.03 

.04 

.01 

Will-Do 

.02 

.11 

.10 

-.08 

.21 

.10 

.09 

.06 

.08 

.07 

Adaptation 

.11 

.06 

.04 

-.04 

.14 

.07 

.08 

.00 

.01 

.00 

Note.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed). Correlations  based  on  fewer  than  50  cases  are  not  reported.  Sample  sizes  range  from  n  =  640  -  944 
"  Sample  sizes  for  six  TAPAS  dimensions  were  considerably  smaller  than  the  other  dimensions  (n  =  2  -  10).  JKT  =  Job  Knowledge  Test.  WTBD  =  Warrior  Tasks  and  Battle  Drills. 
PRS  =  Performance  Rating  Scales.  ALQ  =  Army  Life  Questionnaire. 
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Table  D.5.  Summary  of  the  Bivariate  Correlations  between  AFQT,  TAPAS,  and  Selected  In-Unit  Criteria  for  Tier  1+2  Soldiers 


WTBD 

JKT 

Can-Do 

(PRS) 

Effort  & 
Personal 
Discipline 
(PRS) 

Disciplinary 

Incidents 

(ALQ) 

Last 

APFT 

Score 

(ALQ) 

Working 

with 

Others 

(PRS) 

Overall 

Leadership 

(PRS) 

Affective 

Commitment 

(ALQ) 

Army  Fit 
(ALQ) 

Reenlistment 

Intention 

(ALQ) 

AFQT 

.37 

.10 

.11 

-.01 

-.03 

.12 

.07 

-.06 

-.06 

-.08 

Individual  TAPAS  Scales 
Achievement 

.04 

.11 

.11 

-.09 

.03 

.09 

.08 

.08 

.10 

.11 

Adjustment 

.07 

.02 

.00 

-.03 

.00 

.01 

-.02 

.01 

.05 

.02 

Adventure  Seeking  a 

- 

- 

- 

- 

- 

- 

- 

— 

- 

— 

Attention  Seeking 

.00 

-.01 

-.03 

.06 

-.01 

-.06 

-.04 

-.01 

.04 

-.06 

Commitment  to  Serve a 

- 

— 

— 

— 

— 

— 

— 

— 

— 

— 

Cooperation 

-.01 

-.05 

-.08 

-.05 

-.04 

-.06 

-.06 

.04 

.05 

.03 

Courage  a 

- 

- 

- 

- 

- 

- 

- 

— 

- 

— 

Dominance 

.03 

.04 

.03 

-.01 

.09 

.03 

.02 

.06 

.08 

.05 

Even  Tempered 

.10 

.00 

-.06 

-.04 

-.04 

.00 

-.01 

.07 

.07 

.07 

Intellectual  Efficiency 

.16 

.05 

.03 

-.03 

.01 

.04 

-.03 

.05 

.05 

.02 

Non-Delinquency 

.00 

.07 

.03 

-.04 

-.08 

.06 

.04 

.08 

.06 

.07 

Optimism 

.00 

.02 

.01 

-.04 

.02 

.02 

.01 

.03 

.07 

.03 

Order 

-.08 

-.06 

-.03 

-.08 

.05 

-.02 

-.03 

.04 

.03 

.05 

Physical  Conditioning 

-.01 

.06 

.06 

-.06 

.26 

.08 

.07 

.00 

.01 

.00 

Responsibility 

- 

- 

- 

- 

- 

- 

- 

— 

- 

— 

Self-Control 

.04 

.02 

.05 

-.06 

.02 

.02 

.02 

.07 

.04 

.07 

Selflessness 

-.05 

-.04 

-.03 

.01 

-.04 

-.03 

-.02 

.04 

.03 

.05 

Situational  Awareness 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

Sociability 

-.09 

.02 

-.02 

.02 

.04 

-.01 

-.02 

.03 

.03 

.02 

Team  Orientation  a 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

Tolerance 

-.02 

.00 

.02 

.04 

.00 

.04 

.01 

.04 

.05 

.03 

TOPS  Composites  (Original) 
Can-Do  .11 

.09 

.04 

-.08 

-.02 

.07 

.03 

.11 

.13 

.11 

Will-Do 

.05 

.11 

.08 

-.11 

.09 

.12 

.09 

.10 

.09 

.13 

TOPS  Composites  (Revised) 

Can-Do  .22 

.07 

.04 

-.03 

-.04 

.05 

.00 

.03 

.04 

.01 

Will-Do 

.02 

.10 

.10 

-.09 

.21 

.10 

.09 

.06 

.09 

.07 

Adaptation 

.11 

.07 

.05 

-.04 

.14 

.07 

.08 

.00 

.02 

.00 

Note.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed). Correlations  based  on  fewer  than  50  cases  are  not  reported.  Sample  sizes  range  from  n  =  650  -  960.  JKT 
=  Job  Knowledge  Test.  WTBD  =  Warrior  Tasks  and  Battle  Drills.  PRS  =  Performance  Rating  Scales.  ALQ  =  Army  Life  Questionnaire. 

"  Sample  sizes  for  six  TAPAS  dimensions  were  considerably  smaller  than  the  other  dimensions  (n  =  2  -  10).  *  Common  TAPAS  Scales  =  TAPAS  dimensions  that  are  included  in 
all  TOPS  composites. 


Table  D.6.  Summary  of  the  Correlations  between  AFQT,  TAPAS,  and  Selected  Attrition 
Criteria  for  Tier  1  Soldiers 


Attrition 


Predictor 

6-Mos 

12-Mos 

18-Mos 

24-Mos 

30-Mos 

AFQT 

Individual  TAPAS  Scales 

-.06 

-.06 

-.07 

-.11 

-.19 

Achievement 

-.02 

-.02 

-.02 

-.03 

-.03 

Adjustment 

-.01 

-.02 

-.02 

-.01 

.00 

Adventure  Seeking  a 

-.01 

.17 

- 

- 

- 

Attention  Seeking 

-.03 

-.03 

-.01 

-.01 

-.02 

Commitment  to  Serve  a 

.03 

.01 

— 

— 

— 

Cooperation 

.00 

.00 

-.01 

.00 

.01 

Courage  a 

.00 

-.19 

- 

- 

- 

Dominance 

-.02 

-.02 

-.02 

-.02 

-.06 

Even  Tempered 

-.01 

-.01 

-.01 

-.01 

.00 

Intellectual  Efficiency 

-.01 

-.01 

-.01 

-.03 

-.04 

Non-Delinquency 

.01 

.01 

.01 

.00 

.00 

Optimism 

-.02 

-.02 

-.02 

-.02 

-.04 

Order 

.01 

.02 

.02 

.02 

.03 

Physical  Conditioning 

-.07 

-.08 

-.07 

-.09 

-.11 

Responsibility 

-.01 

-.07 

- 

- 

- 

Self-Control 

.00 

.00 

.00 

-.01 

.00 

Selflessness 

.03 

.03 

.03 

.04 

.01 

Situational  Awareness 

-.03 

-.04 

— 

- 

- 

Sociability 

-.01 

.00 

.02 

.03 

.03 

Team  Orientation  a 

-.05 

-.15 

— 

- 

- 

Tolerance 

.01 

.01 

.01 

.01 

.01 

TOPS  Composites  (Original) 
Can-Do 

-.01 

-.02 

-.02 

-.03 

-.04 

Will-Do 

-.02 

-.03 

-.04 

-.04 

-.05 

TOPS  Composites  (Revised) 
Can-Do 

-.02 

-.02 

-.03 

-.05 

-.06 

Will-Do 

-.06 

-.07 

-.06 

-.08 

-.10 

Adaptation 

-.06 

-.07 

-.08 

-.09 

-.11 

Note.  Estimates  are  the  observed  point-biserial  correlation  (rpb)  between  Soldiers'  predicted  probability  of  an  event  (e.g.,  attrition, 
graduating  IMT  without  a  restart)  and  their  actual  behavior.  Large,  positive  rpb  values  mean  that  the  TOPS  composite  or  scale 
positively  predicted  Soldiers’  actual  behavior.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed).Correlations  based 
on  fewer  than  50  cases  are  not  reported.  6-Mos  n  =  55,683  -  58,91 1 ;  12-Mos  n  =  42,342-42,526  - ;  18-Mos  n  =  29,962  -29,884;  24- 
Mos  n  =  15,980  -  15,906;  30-Mos  n  =  5,095  -  5,064. 

a  Sample  sizes  for  six  TAPAS  dimensions  were  considerably  smaller  than  the  other  dimensions  (n  =  19  -  1,622). 
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Table  D.7.  Summary  of  the  Correlations  between  AFQT,  TAP  AS,  and  Selected  Attrition 
Criteria  for  Tier  2  Soldiers 


Attrition 


Predictor 

6-Mos 

12-Mos 

18-Mos 

24-Mos 

30-Mos 

AFQT 

Individual  TAP  AS  Scales 

-.01 

-.05 

-.08 

-.09 

-.10 

Achievement 

-.02 

-.03 

.04 

.03 

.02 

Adjustment 

-.03 

-.04 

.08 

.00 

.03 

Adventure  Seeking  a 

.00 

- 

- 

- 

- 

Attention  Seeking 

-.04 

-.05 

-.06 

-.11 

-.08 

Commitment  to  Serve  a 

.10 

— 

— 

— 

— 

Cooperation 

.02 

.05 

.06 

-.07 

-.04 

Courage  a 

-.06 

- 

- 

- 

- 

Dominance 

.00 

-.03 

.02 

-.01 

.03 

Even  Tempered 

.02 

.03 

.06 

.04 

.06 

Intellectual  Efficiency 

.00 

.00 

.09 

.03 

.02 

Non-Delinquency 

.03 

.00 

-.03 

.08 

.06 

Optimism 

-.03 

-.04 

-.01 

-.03 

-.02 

Order 

-.01 

-.01 

-.05 

-.09 

-.11 

Physical  Conditioning 

-.03 

-.05 

.03 

.09 

.10 

Responsibility 

.13 

- 

- 

- 

- 

Self-Control 

.03 

.09 

.24 

.26 

.23 

Selflessness 

.04 

.02 

.04 

.00 

-.02 

Situational  Awareness 

.00 

— 

- 

— 

— 

Sociability 

.00 

-.01 

-.04 

-.08 

-.06 

Team  Orientation  a 

.07 

— 

— 

— 

— 

Tolerance 

.01 

-.02 

.09 

.01 

-.01 

TOPS  Composites  (Original) 

- 

- 

- 

- 

Can-Do 

.00 

-.02 

.03 

.02 

.00 

Will-Do 

.02 

.00 

.07 

.17 

.16 

TOPS  Composites  (Revised) 

- 

- 

- 

- 

Can-Do 

.01 

.01 

.09 

.08 

.07 

Will-Do 

-.03 

-.06 

.04 

.06 

.08 

Adaptation 

-.01 

-.01 

.10 

.20 

.22 

Note.  Estimates  are  the  observed  point-biserial  correlation  (rpb)  between  Soldiers'  predicted  probability  of  an  event  (e.g.,  attrition, 
graduating  IMT  without  a  restart)  and  their  actual  behavior.  Large,  positive  rpb  values  mean  that  the  TOPS  composite  or  scale 
positively  predicted  Soldiers’  actual  behavior.  Correlations  in  bold  are  statistically  significant  (p  <  .01,  two-tailed). Correlations 
based  on  fewer  than  50  cases  are  not  reported.  6-Mos  n  =  1,620  -  1,827;  12-Mos  n  =  827  -  835;  18-Mos  n  =  226  -  233; 

24-Mos  n  =  106  -  113;  30-Mos  n  =  101  -  108. 

a  Sample  sizes  for  six  TAP  AS  dimensions  were  considerably  smaller  than  the  other  dimensions  (n  =  1  -  103). 


D-7 


Table  D.8.  Summary  of  the  Correlations  between  AFQT,  TAPAS,  and  Selected  Attrition 
Criteria  for  Tier  1+2  Soldiers 


Attrition 

Predictor 

6-Mos 

12-Mos 

18-Mos 

24-Mos 

30-Mos 

AFQT 

-.06 

-.06 

-.07 

-.11 

-.19 

Individual  TAPAS  Scales 
Achievement 

-.02 

-.02 

-.02 

-.03 

-.03 

Adjustment 

-.01 

-.02 

-.02 

-.01 

.00 

Adventure  Seeking  a 

-.01 

.17 

- 

- 

- 

Attention  Seeking 

-.03 

-.03 

-.01 

-.01 

-.03 

Commitment  to  Serve  a 

.04 

.00 

— 

— 

— 

Cooperation 

.00 

.00 

-.01 

.00 

.01 

Courage  a 

.00 

-.24 

- 

- 

- 

Dominance 

-.02 

-.02 

-.02 

-.02 

-.06 

Even  Tempered 

.00 

.00 

-.01 

-.01 

.00 

Intellectual  Efficiency 

-.01 

-.01 

-.01 

-.03 

-.04 

Non-Delinquency 

.01 

.01 

.01 

.00 

.00 

Optimism 

-.02 

-.02 

-.02 

-.02 

-.03 

Order 

.01 

.02 

.02 

.02 

.03 

Physical  Conditioning 

-.06 

-.08 

-.07 

-.09 

-.11 

Responsibility 

.00 

-.02 

- 

- 

- 

Self-Control 

.01 

.00 

.00 

.00 

.00 

Selflessness 

.03 

.03 

.03 

.04 

.01 

Situational  Awareness 

-.03 

-.04 

— 

— 

— 

Sociability 

-.01 

.00 

.02 

.02 

.02 

Team  Orientation  a 

-.04 

-.13 

— 

— 

— 

Tolerance 

.01 

.01 

.02 

.01 

.01 

TOPS  Composites  (Original) 
Can-Do 

-.01 

-.02 

-.02 

-.03 

-.04 

Will-Do 

-.02 

-.03 

-.04 

-.04 

-.04 

TOPS  Composites  (Revised) 
Can-Do 

-.01 

-.02 

-.03 

-.05 

-.06 

Will-Do 

-.06 

-.07 

-.06 

-.07 

-.10 

Adaptation 

-.06 

-.07 

-.08 

-.09 

-.11 

Note.  Estimates  are  the  observed  point-biserial  correlation  (rph)  between  Soldiers'  predicted  probability  of  an  event  (e.g.,  attrition, 
graduating  IMT  without  a  restart)  and  their  actual  behavior.  Large,  positive  rpb  values  mean  that  the  TOPS  composite  or  scale 
positively  predicted  Soldiers’  actual  behavior.  Correlations  in  bold  are  statistically  significant  {p  <  .01,  two-tailed).Correlations  based 
on  fewer  than  50  cases  are  not  reported.  6-Mos  n  =  57,303  -  60,738;  12-Mos  n  =  43,169  -  43,361;  18-Mos  n  =  30,1 10  -  30,195; 
24-Mos  n  =  16,012  -  16,093;  30-Mos  n  =  5,127  -  5,203. 

“  Sample  sizes  for  six  TAPAS  dimensions  were  considerably  smaller  than  the  other  dimensions  ( n  =  24  -  1,725). 
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